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I.  INTRODUCTION 


In  education  It  Is  crucial  to  access  objectively  and 
accurately  a  student's  level  of  proficiency.  Computer-Aided 
Testing  (CAT)  Is  becoming  one  of  the  most  effective 
evaluation  tools  available  to  the  educator.  The  greatest 
challenge  to  developing  computer-aided  tests  is  modeling  a 
testing  session  in  an  intelligent  manner. 

Most  current  computer-aided  tests  are  based  on 
enumerative  or  adaptive  methods  (Bejar  1986,  Thomas  1986, 
Larson  19871.  Enumerative  testing  is  similar  to  traditional 
written  examinations  in  which  an  individual  is  asked  a  fixed 
number  of  predetermined  questions.  Adaptive  testing  on  the 
other  hand  selects  both  the  number  and  type  of  questions 
presented  to  the  individual  according  to  his  personal  skills 
and  previously  aquired  knowledge. 

This  thesis  proposes  and  validates  a  computer-aided 
testing  model  called  the  Interrogative  Diagnostic  Model 
( I DM ) .  The  model  is  both  declarative  and  adaptive  in 
nature.  In  other  words,  a  pool  of  questions  are  explicitly 
stored  in  a  test  bank  along  with  their  difficulty  level. 
The  model  uses  an  algorithm  based  on  Bayesian  techniques  to 
determine  both  the  number  and  the  difficulty  level  of 
questions  to  present.  The  power  of  the  model  lies  in  its 
ability  to  automatically  adapt  a  testing  session  to  the 
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Individual's  level  of  knowledge  by  choosing  the  domain,  type 
and  number  of  questions. 

The  model  was  implemented  as  a  computer  program  and  was 
validated  in  an  experiment  with  34  subjects.  Analysis  of  the 
results  indicated  that  the  IDM  can  evaluate  student 
knowledge  as  effectively  as  traditional  written  examinations 
but  with  a  significantly  less  number  of  questions  and 
shorter  test  duration.  Chapter  II  of  the  paper  reviews 
related  literature  in  the  area  of  computer  aided  adaptive 
testing.  Chapter  III  desribes  the  IDM  in  detail.  The 
subsequent  chapter  discusses  the  computer  implementation  of 
the  model.  Design  and  administration  of  the  experiment  is 
described  in  Chapter  V.  The  statistical  analysis  of  the 
results  are  presented  in  Chapter  VI.  Chapter  VI I  concludes 
by  stating  the  advantages  and  disadvantages  of  the  model  and 
suggesting  directions  for  future. 


II.  LITERATURE  REVIEW 


Although  considerable  work  has  been  done  in  the  area  of 
computer -aided  instruction,  research  in  computer-aided 
adaptive  testing  has  been  sparse  [Brown  1977,  Clancey  1984, 
Clancey  1988,  Davis  1983].  Weiss  [Weiss  1977]  described  a 
number  of  strategies  for  adapting  test  items  to  individuals 
differing  abilities.  Tennyson  and  Ruthen  [Tennyson  1977] 
verified  that  an  adaptive  strategy  in  selecting  the  number 
of  Instances  needed  to  learn  concepts  was  more  efficient 
than  either  partially  adaptive  or  non-adaptive  strategies. 
Further  evidence  that  adaptive  testing  has  validity  was 
presented  in  an  experiment  by  Thathon  and  Kruavan  [Thathon 
1985]  in  which  a  self  scoring,  flexi-level  adaptive  testing 
procedure  proved  both  effective  and  practical. 

Bayesian  statistical  techniques  have  also  been 
investigated  in  the  adaptive  testing.  Sympson  [Sympson 
1978]  applied  Bayesian  techniques  to  measure  ability  of 
candidates  using  a  hypothetical  20  item  adaptive  test. 
Bayesian  analysis  has  also  been  applied  to  illustrate  the 
basic  mathematical  concepts  involved  in  scoring  multiple 
choice  achievement/aptitude  tests  [Warm  1978,  Vale  19811. 
Purther  application  of  Bayesian  analysis  to  adaptive  testing 
vas  done  by  Sivasankaran  and  Bui  [Sivasankaran  19861.  Their 
work  provides  a  basis  for  the  I  DM. 
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Methods  of  interpeting  a  candidate's  response  and 
Immediate  modification  of  the  test  itself  were  discussed  by 
McArthur  and  Chou  [McArthur  1984].  Loyd  [Loyd  1984] 
presented  evidence  that  a  tvo-stage  adaptive  testing 
procedure,  in  which  the  first  stage  routes  the  individual  to 
second  stage  which  focuses  on  the  individuals  ability  level 
is  a  more  precise  measure  than  a  single  test.  Be jar  [Be jar 
1986]  describes  a  further  development  in  adaptive  testing  in 
which  the  testing  program  not  only  decides  the  number  of 
questions  but  also  actually  generates  new  questions  based  on 
the  ability  of  the  person  being  tested. 

However  no  attempt  has  been  made  to  investigate  the 
effects  of  incorporating  learning  theory  into  computer-aided 
adaptive  testing.  The  Interrogative  diagnostic  model 
presented  in  this  paper  makes  use  of  Bayesian  techniques  and 
incorporates  a  multi-stage  testing  procedure  based  on 
cognitive  learning  theory  proposed  by  Kibler  et  al  [Kibler 
1970].  The  experiment  conducted  using  the  IDM  was  patterned 
after  the  design  of  Thathon  and  Kruawan  (Thathon  1985]  in 
that  a  group  of  individuals  was  given  both  a  written  and  a 
computer  test  covering  the  same  topics  and  the  results  of 
the  exams  compared. 
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III.  THE  INTERROGATIVE  DIAGNOSTIC  MODEL 

A.  INTERROGATIVE  LEARNING  MEASUREMENT 

According  to  the  learning  theory  proposed  by  Klbler  et 
al.  (Klbler  19701/  an  individual  progresses  through 
different  levels  of  learning  (Table  1).  Levels  1  and  2  fora 
the  first  phase  in  which  the  individual  begins  to  recall  or 
recognize  information  and  be  able  to  translate  it  from  one 
fora  to  another.  These  two  levels  draw  heavily  on  the 
ability  to  memorize  information.  During  the  second  phase  of 
the  learning  process,  levels  3  through  6,  the  Individual 
assimilates  and  Integrates  information  of  increasing 
complexity. 

The  I DM  has  accordingly  two  types  of  questions  which 
directly  correspond  to  the  these  two  phases  of  learning. 
These  are  the  Supplementary  and  Critical  questions. 
Supplementary  questions  are  those  questions  which  are  basic 
in  nature  and  cover  levels  1  &  2.  Critical  questions  are 
those  questions  which  are  advanced  in  nature  and  cover 
levels  3-6. 

The  rationale  for  the  Interrogative  Diagnostic  Model 
( IDM)  of  testing  is  based  on  the  dynamics  of  an  oral 
examination.  Before  an  oral  examination,  the  examiner 
prepares  a  set  of  topics  to  be  covered  and  within  them 
identifies  specific  questions.  But  the  difficulty  level  and 
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number  of  questions  actually  selected  and  presented  during 
the  examination  are  guided  by  the  student's  answers  to 
previous  questions.  This  type  of  Interaction  can  best  be 
modeled  by  incorporating  Bayesian  techniques.  The  I DM  makes 
use  of  these  techniques. 

TABLE  1  COGNITIVE  LEARNING  LEVELS  iKibler  1970) 

LEVEL  (Lowest  -  1,  Highest  -  2) 

1  KNOWLEDGE — recall  or  recognize  information 

2  COMPREHENSION — translate  it  from  one  form  to  another 

3  APPLICATION — apply  a  known  procedure  to  a  new 

situation 

4  ANALYSIS — break  down  information 

5  SYNTHESIS — put  information  together 

6  EVALUATION — evaluate  something  according  to  some 

criterion 

B.  MODEL  EXECUTION  STEPS 

Figure  1  illustrates  the  model  in  the  form  of  a 
flowchart.  The  model  first  formulates  a  hypothesis  about  a 
student's  proficiency  in  a  given  topic  area  and  then 
evaluates  that  hypothesis  using  a  bank  of  questions.  The 
model  then  derives  conclusions  about  the  student's 
knowledge . 
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Plgura  1  I DM  Plov  Chart 
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The  I DM  consists  of  eight  steps: 

1.  A  topic  area  is  selected. 

2.  A  hypothesis  is  Bade  that  the  student  is  more  likely 
an  average  student  and  a  prior  probability  value 
Initialized. 

3.  A  question  is  presented  to  the  student. 

4.  The  student  response  is  evaluated  and  the  earlier 
probability  is  updated. 

5.  The  probability  is  compared  with  evaluation  criteria 
and  further  questions  are  asked  if  the  criteria  are 
not  aet. 

6.  The  hypothesis  is  accepted  or  rejected. 

a.  If  the  hypothesis  is  rejected  it  is  because  the 
student  is  either  above  or  belov  average  and  proceeds 
to  step  8.  In  either  case  the  model  selects  a  nev 
topic  area  In  which  to  query  the  student. 

b.  If  the  hypothesis  is  accepted  it  is  because  the 
student  is  average  and  the  model  proceeds  to  step  7. 

7.  A  decision  is  made  to: 

a.  end  the  test  if  all  topic  areas  have  been  covered. 

b.  select  supplemental  questions  if  the  critical 
questions  have  been  asked. 

c.  select  the  next  topic  if  the  supplemental 
questions  have  been  asked. 

8.  A  decision  is  made  to: 

a.  end  the  test  If  all  topic  areas  have  been  covered. 

b.  select  the  next  topic  area. 

The  Bayesian  theory  as  applied  in  the  interrogative 
diagnostic  model  steps  shown  above  is  presented  in  detail  in 
Appendix  A. 


C.  BAYESIAN  PROBABILITY  AND  QUESTION  ORDERING 

Since  Bayesian  probabilities  are  affected  by  the  ongoing 
performance  of  the  individual,  the  ordering  of  the  different 
questions  selected  during  the  testing  process  can 
significantly  affect  the  judgement  about  the  students 
proficiency.  Three  general  methods  of  question  ordering  are 
considered:  decreasing  difficulty,  increasing  difficulty  and 
alternating  sequence.  If  questions  are  presented  in  a 
decreasing  order  of  difficulty,  an  average  student 
encountering  the  aore  difficult  questions  at  the  beginning 
of  a  testing  session  aight  be  inappropriately  underevaluated 
since  he  would  reach  the  cutoff  threshold  probability  early. 
On  the  other  hand,  if  the  questions  are  presented  in  an 
increasing  order  of  difficulty,  the  average  student  aight 
tend  to  be  judged  brighter  than  he  really  is.  A  third 
method  is  to  order  questions  in  an  alternating  sequence  of 
difficulty.  Theoretically,  several  other  methods  are  also 
possible . 

The  effect  of  the  three  orderings  on  the  student 
evaluation  is  plctorially  in  Figure  2.  The  first  graph 
shows  the  effect  of  a  decreasing  order  of  difficulty.  Proa 
this  graph  it  can  be  seen  that  the  aovlng  average  of  the 
difficulty  does  not  aove  down  as  rapidly  as  the  actual 
difficulty.  Regions  I,  II,  and  III  of  the  graph  represent 
the  level  of  difficulty  that  a  poor,  average  and  superior 
student  could  reasonably  be  expected  to  handle  respectively. 


In  the  first  graph,  the  average  difficulty  level  does 
not  fall  into  the  average  region  until  the  fourth  question 
has  been  asked.  By  this  tiae  the  Bayesian  calculations  aay 
have  already  concluded  that  the  student  is  poor  when  in 
reality  he  is  average. 

In  a  similar  fashion  the  second  graph,  shoving  an 
increasing  order  in  difficulty  level  might  overrate  an 
average  student.  In  this  case  the  average  difficulty  level 
is  belov  the  actual  level  and  only  when  the  fourth  question 
is  reached  does  the  average  difficulty  level  move  into  the 
region  where  the  average  student  would  be  challenged.  By 
the  fourth  question  the  Bayesian  calculations  may  have 
concluded  that  the  student  is  superior  when  he  could  be 
average . 

The  alternating  sequence  shown  in  the  third  graph 
overcomes  the  limitations  of  the  previous  orderings  by 
alternating  the  question  level.  As  can  be  seen  the  average 
difficulty  level  quickly  moves  into  the  average  area  and 
remains  there  with  ever  smaller  oscillations.  By  remaining 
in  the  average  area  there  is  less  likelihood  that  an  average 
student  will  be  under  or  over  evaluated. 
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Implementation  of  the  interogative  diagnostic  aodel  vas 
divided  into  two  main  areas:  prograa  design  and  question 
design.  Prograa  design  vas  closely  allied  vith  the  first 
assumption  and  Bayesian  statistical  techniques  presented 
earlier.  The  area  of  question  design  corresponded  closely 
with  the  second  assumption  that  questions  can  be  vrltten 
vhich  test  different  levels  of  the  cognative  domain. 

A.  PROGRAM  DESIGN 

The  design  of  a  prograa  to  implement  the  model  vas  quite 
straight  forvard.  dBASB  III  Plus  vas  the  language  selected 
for  implementation  since  the  questions  vere  easily  stored 
and  manipulated  in  a  database.  Because  the  specifications 
and  structure  of  the  prograa  vere  not  fixed,  a  prototyping 
approach  vas  used  in  designing  and  vrltlng  the  program.  The 
final  program  consists  of  eight  modules  vhose  data  flov 
diagram  is  shovn  in  Figure  3. 

The  MAIM  module  retrieves  criteria  to  be  used  in  the 
Bayesian  calculations,  opens  data  files,  selects  topic 
areas,  and  calls  subordinate  modules.  The  IMFO  module 
presents  the  test  instructions  and  gets  student 
ldentlfcatlon  information.  The  ASK  module  presents 
questions  to  the  student  by  calling  the  DOIT  and  QUB8TI0M 


modules  In  the  case  of  critical  questions  or  from  data  files 
In  the  case  of  supplemental  questions. 

The  ASK  module  also  forwards  student's  response  to  the 
CORRECT  module  for  updating  posterior  probabilities.  The 
PACTOR  module  calculates  factorials  for  the  CORRECT  module 
and  the  TALLY  module  writes  the  question  identicatlon 
number,  student  answer  and  time  taken  to  a  data  file. 

The  program  evaluates  the  student  In  each  topic  area 

using  the  Bayesian  technique  presented  in  Appendix  A.  The 

numerical  scores  given  by  the  computer  program  for  each 

topic  range  as  follows: 

5  Superior 
4  Above  average 
3  Average 
2  Belov  average 
1  Poor 

A  pictorial  representation  of  the  scoring  process  is  shown 
in  Figure  4.  The  student's  total  score  for  the  entire  exam 
is  an  average  of  the  scores  in  each  topic  area. 

upon  completion  of  coding  the  program  was  compiled  using 
CLIPPER.  This  conversion  allowed  the  program  to  execute 
appreciably  faster  and  eliminated  the  the  necessity  for  the 
student  to  begin  the  exam  by  first  invoking  dBASE  III  Plus. 

B.  QUESTION  DESIGN 

In  designing  questions  consideration  was  given  to 
question  type,  content,  cognitive  level,  ordering,  and 
presentation.  The  types  of  questions  used  in  the  computer 


exaa  were  Halted  to  aultlple  choice  and  true/false.  Due  to 
the  inability  of  the  dBASE  III  Plus  in  processing  natural 
language  essay  type  questions  were  excluded.  The  content  of 
the  questions  caae  froa  the  textbook  instructors  uanual, 
past  exaainations  and  the  instructor.  The  questions  were 
categorized  by  the  Instructor  into  critical  and 
suppleaentary  as  discussed  in  Chapter  III.  However,  Instead 
of  assigning  a  specific  difficuly  aeasure  to  each  question, 
each  question  within  a  catagory  was  assigned  one  of  three 
difficulty  levels,  with  Level  1  being  the  least  difficult 
and  Level  3  being  the  aost  difficult.  The  text  of  both  the 
critical  and  suppleaental  questions  used  in  the  prograa  are 
presented  In  Appendix  C.  The  alternating  sequence  of 
difficulty  discussed  in  Chapter  III  was  used  to  order  the 
questions  within  a  cognitive  level. 

Presentation  of  the  questions  to  the  subjects  was 
challenging.  Many  of  the  critical  questions  were  too  long 
to  fit  on  the  standard  80  x  25  character  screen.  This 
problea  was  solved  by  partitioning  each  critical  question 
into  two  prograa  files.  Dividing  the  critical  questions 
into  two  screens  aeant  that  the  students  were  required  to 
page  back  and  forth  betveen  screens  to  view  a  single 
question. 


V.  RESEARCH  DESIGN 


A.  SETTING 

A  aicro-coaputer  lab  equipped  vith  25  IBM  PCs  was  used 
for  adalnistration  of  the  coaputer  exaa.  A  diskette 
containing  the  coaplled  version  of  the  computer  exaa  was 
given  to  each  student.  A  veek  prior  to  the  actual  exaa  a 
deaonstratlon  exaa  was  given  to  the  student  In  the  saae  lab 
in  order  to  let  the  student  becoae  familiar  with  the  testing 
prograa.  To  further  ensure  that  the  students  did  not 
encounter  any  technical  probleas  a  facilitator  was  present 
during  the  exaaination. 

B .  SUBJECTS 

The  subjects  selected  vere  aasters  degree  aanageaent 
lnforaation  systeas  students  enrolled  in  a  database 
aanageaent  course  at  the  Naval  Postgraduate  School.  The 
subjects  vere  of  a  hoaogenlus  background;  all  vere  junior 
ailitary  officers  in  their  third  quarter  in  the  coaputer 
technology  prograa  The  course  vas  divided  into  tvo  sections 
of  21  and  13  Individuals  respectively,  and  the  saae 
instructor  taught  both  sections. 

C.  EXAM  DESIGN 

The  subject  area,  database  aanageaent,  vas  divided  into 
three  topic  areas,  basic  concepts,  advanced  concepts  and 
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methodologies  (normalization),  and  issues  related  to 
management  of  databases.  Each  topic  area  had  S  critical  and 
20  supplementary  questions.  Both  the  written  and  computer 
final  examinations  consisted  of  the  same  75  questions  (15 
critical  and  60  supplemental)  and  both  had  a  two  hour  time 
limit.  The  written  examination  presented  all  75  questions 
to  the  subjects  and  the  questions  were  not  grouped  according 
to  topic  area  or  cognitive  level.  The  computer  exam 
presented  a  varying  number  of  questions  to  the  subject 
according  to  the  Bayesian  evaluation  of  the  subject's 
previous  answers.  In  the  computer  exam  questions  were 
grouped  according  to  topic  area.  Upon  finishing  the  *xam 
each  student  was  given  a  questionnaire  to  determine  his 
reaction  to  the  exam. 

In  order  to  compensate  for  any  learning  dynamics  which 
could  have  influenced  the  responses  from  the  first  test  to 
the  second  test,  the  experiment  was  conducted  in  the 
following  manner.  One  section  was  given  the  computer-aided 
exam  first,  followed  by  the  written  exam.  The  other  section 
was  given  the  written  exam  first  followed  by  the  computer 
exam. 

Upon  finishing  the  exam  each  student  was  given  a 
questionnaire  to  determine  his  reaction  to  the  exam  and  to 
attempt  to  determine  any  other  factors  which  might  have 
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influenced  his  score.  The  questionaire  is  presented  in 
Appendix  C. 

0.  HYPOTHESES 

Based  on  the  premise  that  recent  developments  in 
computer-aided  testing  tools  would  effectively  evaluate 
students,  the  goal  of  the  experiment  was  to  focus  on 
determining  how  such  tools  compare  with  traditional  written 
examinations.  More  specifically,  this  experiment  was 
designed  to  evaluate  the  validity  of  the  Interrogative 
Diagnostic  Model.  The  following  hypotheses  were  tested: 

HI:  The  scores  achieved  by  a  student  will  be  the  same 
Irrespective  of  whether  the  student  takes  the  written 
or  computer  tests. 

H2:  The  evaluation  quality  both  of  the  computer  and  of 
the  written  exam  will  be  the  same. 

H3:  The  number  of  questions  presented  to  the  student  in 
the  computer  exam  will  be  the  same  as  the  number  of 
questions  in  the  written  exam. 

H4:  The  time  taken  to  complete  the  computer  exam  will  be 
the  same  as  the  time  taken  to  complete  the  written 
exam. 

E.  HYPOTHESIS  TESTING  METHODOLOGY 

In  testing  the  above  hypotheses,  a  factor  analysis  using 
the  student's  computer  score,  section  and  responses  to  the 
questionaire  was  performed  to  identify  dominant  factors.  A 
regression  analysis  was  done  with  these  factors  as 
independent  variables  against  the  written  exam  score  as  the 


dependent  variable.  In  testing  HI,  the  significance  level  of 
the  coefficient  of  the  coaputer  score  term  was  used.  In 
addition,  the  written  exaa  scores  were  converted  to  the 
coaputer  scale  and  a  difference  in  aeans  test  between  the 
coaputer  and  written  exaa  scores  was  conducted. 

The  evaluation  quality  (H2)  for  both  the  written  final 
exaa  and  the  coaputer  final  exaa  was  be  aeasured  by 
coaparing  these  scores  with  the  average  score  of  the 
student's  two  previous  written  exaainations.  This  was 
considered  an  appropriate  aeasure  because  the  entire  course 
consisted  of  three  exaainations  with  the  third  and  final 
exaainatlon  being  coaprehenslve  having  only  about  10  percent 
of  new  aaterlal. 

Since  the  coaputer  exaainatlon  could  theoretically 
present  the  saae  nuaber  of  questions  as  the  written 
exaainatlon,  it  was  expected  that  the  nuaber  of  questions 
presented  and  tlae  taken  for  the  coaputer  exaa  would  be  the 
saae  as  those  of  the  written  exaa.  H3  was  tested  by  counting 
the  nuaber  of  questions  in  the  coaputer  and  written 
exaainations  and  the  two  counts  were  coapared  using  a 
diffenence  in  aeans  test. 

In  testing  H4,  the  tiaes  taken  by  the  student  for  the 
coaputer  exaa  and  for  the  written  were  aeasured  and  coapared 
using  a  difference  in  aeans  test. 
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VI .  UULIfilfi  QE.  BXP1BIM1HT  results 


The  computer  program  automatically  saved  the  exam 
responses,  the  computed  score  and  the  time  taken  by  each 
student.  In  performing  the  factor  analysis,  eight  variables 
vere  Included  to  capture  each  of  the  following:  1)  the 
computer  score  ( C0MP8C0B ) ,  2)  the  section  (8BCTIOM), 
3) Inability  to  explain  answers  (XPLMAN8),  4)  lack  of  essay 
questions  (NOS88YQ),  5)  comfort  level  In  using  computers 
( COMFLVL ) ,  6)  percleved  effectiveness  (PCVDIFCT) ,  7) 
perceived  pressure  (PCVDPRBS),  8)  general  feelings  toward 
computer  testing  (GKNLATT). 

The  factor  extraction  technique  of  principle  axis 
factoring  with  varlmax  rotation  was  used.  Items  which 
failed  to  demonstrate  convergent  and  discriminate  ability  by 
loading  cleanly  on  a  factor  vere  deleted.  The  results  of 
this  analysis  are  reported  In  Table  A.  The  Items  loaded  on 
four  factors.  They  vere  the  computer  score,  the  section, 
perceived  effectiveness,  and  general  discomfort  with  the 
concept  of  computer  testing. 


TABLE  2.  VAR I MAX  ROTATED  FACTOR  MATRIX 


Variable/Factor  1 

2 

3 

4 

SECTION 

-0.32772 

mwrni 

mssBt 

COMP  8  COR 

0.05631 

iiKwT  'TtS 

Ml  Sna 

GENLATT 

0.86687 

0.04518 

-0.12319 

0.04472 

XPLMAMS 

0.56898 

-0.36634 

0.15509 

-0.46568 

NOESSYQ 

0.65517 

-0.07445 

0.63430 

0.06525 

PCVDPRES 

0.79692 

0.26031 

-0.16654 

0.11812 

COMFLVL 

0.09884 

0.66550 

-0.04637 

0.35176 

PCVDBFCT 

0.10792 

0.04534 

0.12618 

0.91199 

considering  only  variables  vlth 

loads  above 

0.80,  It 

can  be  seen  that  factor  1 

loads  on  general  feelings  towards 

coaputer  testing,  factor 

2  on  the  coaputer  score 

,  factor  3 

the  section  and  factor  4 

on  the  perceived  effectiveness  of 

coaputer -aided 

exaas . 

Using  the 

scores  of  these  four  factors  as 

independent 

variables,  a 

aultlple  regression  was 

run  against  the  the 

written  exaa 

scores.  The 

results  of 

this  regression  are 

shown  In  Table 

3. 

TABLE 

3a.  REGRESSION  OF  THE  FACTORS  AGAINST 

WRITTEN  EXAM 

SCORE 

independent 

coefficient 

std.  error 

t -value 

sig. level 

variable 

CON8TAMT 

60.766667 

1.359042 

44.7129 

0.0000 

FACTOR 1 

0.868266 

1.382275 

0.6281 

0.5356 

FACTOR 2 

7.751186 

1.382275 

5.6076 

0.0000 

FACTOR 3 

1.52467 

1.382275 

1.1030 

0.2605 

FACTOR 4 

-1.42072 

1.382275 

-1.0275 

0.3140 

R-SQ.  (ADJ.)  - 

0.5094 

8E-7. 443760 

MAE  -  5.356040 

DurbWat* 

1.959 
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TABLE  3b.  ANALYSIS  OF  VARIANCE  FOR  THE  FULL  RBORE8BION 


Source  8ua 

of  Squares 

DF 

Mean  Square 

F-Ratlo 

P-value 

Model 

090.12 

4 

472.530 

6.52791 

.0002 

Error 

1385.25 

25 

55.4099 

Total  (Cor.) 

3275.37 

29 

R-squared  *  0.577071 
8tnd.  error  of  est.  ■  7.44370 
R-squared  (Adj.  for  d.f.)  *  0.509403 
Durbin-Watson  statistic  ■  1.95059 


Froa  the  table  it  is  evident  that  factor  2  (computer 
score)  is  the  only  significant  factor.  A  separate  regression 
vas  therefore  run  using  the  coaputer  score  as  the  only 
Independent  variable.  Results  of  this  regression  are  shown 
in  Table  4. 

TABLE  4a.  REGRESSION  OF  COMPUTER  SCORE  AGAIM8T 

WRITTEN  EXAM  SCORE 


Independent 

variable 

coefficient 

std.  error 

t-value 

slg. level 

COM 8T ANT 

32.075809 

3.67456 

8.7292 

0.0000 

COMP8COR 

10.172458 

1.251616 

8.1275 

0.0000 

R-8Q.  (ADJ.)  - 
MAE  -  4.970321 

0.6635 

8B  -  6.025962 

DurbWat  -  2.076 

TABLE  4b.  AMALY8X8  OF  VARIENCB  FOR  THE  FULL  REGRESSION 


Source  Sun 

of  Squares 

DF 

Mean  Square 

F-Ratio 

P-value 

Model 

2398.63 

1 

2398.63 

66.0556 

.0000 

Error 

1161.99 

32 

36.3122 

Total  (Corr.) 

3560.62 

33 

R-squared  -  0.673655  R-squared  (Adj.  for  d.f.)  ■  0.663456 
8tnd.  error  of  est.  ■  6.02596 
Durbln-Watson  statistic  ■  2.07571 
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TABLE  (a.  REGRESSION  OF  COMPUTER  SCORE  AGAINST  WRITTEN 
EXAM  SCORE  AFTER  DATA  TRANSFORMATION  AND 
DISREGARDING  CONSTANT  TERM 


Independent  coefficient  std.  error  t-value  aig. level 


COMP8CORO 

12.110138 

0 

.532355  22.7483 

0.0000 

R-SQ.  (ADJ.) 

-  0.9401 

8E  -  6.196253 

MAE  -  5.002530 

DurbWat  ■  1.994 

TABLE  6b. 

AMALY8I8  OF 

VARIEHCB  FOR  THE  FULL  REGRE88ION 

Source  8ua 

i  of  Squares 

DF 

Mean  Square  F-Ratlo 

P-value 

Model 

19668.0 

1 

19866.0  517.463 

.0000 

Error 

1266.99 

33 

38.39 

Total 

21135.0 

34 

R-squared  »  0.940053 
8tnd.  error  of  eat.  ■  6.19625 
R-aquared  (Adj.  for  d.f.)  *  0.940053 
Durbin-Wataon  statistic  *  1.99362 


The  the  high  values  of  R-aquared,  the  F-ratlo  and  the 
lov  nean  square  error  tend  to  confirn  that  it  is  sound  to 
aodel  the  coaputer  score  aa  an  laportant  factor  Influencing 
the  written  exaalnatlon  score.  The  difference  in  Beans  test 
was  also  perforaed  using  the  transforaed  data  (Table  7).  It 
can  be  seen  that  HI  cannot  be  rejected. 


TABLB  7.  TW0-8AMPLB  ANALYSIS  KBSULT8 


Saaple  1 

Saaple  2 

Pooled 

Saaple 

Nuaber  of  Obs. 

34 

34 

68 

Statistics: 

Average 

2.99149 

2.90776 

2.94963 

Variance 

1.86803 

1.7982 

1.83311 

Std.  Deviation 

1.36676 

1.34097 

1.35393 

Median 

2.89474 

3.2 

3.11316 

Difference 

between  Means  ■  i 

0.0837214 

Conf.  Interval  Por  Diff.  in 

Means: 

95  Percent 

(Bqual  vara.)  Sap 11  -  Sap 12  -0.572049  0.739492  66  D.P. 
(Unequal  Vara.)  Sapll  -  Sap 12  -0.572054  0.739496  66.0  D.P. 

Ratio  of  Parlances  -  1.03884 

Conf.  Interval  for  Ratio  of  Variances:  0  Percent 

Saaple  1  v  Saaple  2 

Hypothesis  Test  for  HO: 

Diff  -  0  Coaputed  t  statistic  *  0.254956 

vs  Alt:  NR  Big.  Level  •  0.799549 

at  Alpha  -  0.05  so  do  not  reject  HO. 


The  second  hypothesis,  that  the  coaputer  exaa  would 
exhibit  the  saae  quality  of  evaluation  as  the  written  exaa, 
was  tested  by  coaparing  both  the  written  exaa  scores  and  the 
coaputer  exaa  scores  with  the  average  of  the  scores  of  two 
previous  exaas  the  students  had  taken.  The  rationale  for 
this  coaparison  was  discussed  previously  in  hypothesis 
testing  aethodology.  The  coabined  results  of  the  two 
previous  exaas,  called  8TUQUAL,  served  as  the  standard  in 
deteraining  quality.  When  the  the  written  and  coaputer 
scores  were  coapared  with  STUQUAL,  the  constant  tera 
generated  was  not  statistically  significant  and  it  was 
considered  appropriate  to  disregard  the  constant  and  force 
the  regression  through  xero.  The  results  of  this  regression 


ace  shown  in  Table  8.  These  results  show  an  R-squared  o£ 


0.9809  for  the 

written 

final 

exan 

and  0.9380  for 

the 

computer  exaa. 

Though 

there 

is  a 

difference,  it 

is 

considered  slight 

,  and  based  on 

these 

results,  H2  cannot 

be 

rejected. 


TABLN  8a.  REGRESSION  OF  WRITTEN  FINAL  SCORN 
AGAINST  PRIOR  AVERAGE  SCORN 


Ind. variable  coefficient 

std. error 

t- value 

sig. level 

PINAL 

1.428209 

0.03483 

41.1837 

0.0000 

R-8Q. 
MAN  - 

(ADJ. )  -  0.9809 
10.019345 

8N  -  12.437048 
DurbWat  ■  1.828 

TABLN  8b 

.  ANALYSIS  OP 

VARIANCE  FOR  THE 

PULL  REGRESS I OM 

8ource 

Sun  of  Squares 

DP 

Mean  Square 

F-Ratio  P-value 

Model 

282353. 

1 

282353. 

1698.10  .0000 

Brror 

5104.44 

33 

154.880 

Total 

34 

287457. 

R-squared 

-  0.980915 

R-squared 

(Adj.  for  d.f 

. )  ■ 

0.980915 

Stnd.  error  of  est.  -  12.437 
Durbln-Vatson  statistic  ■  1.82842 


TABLN  8c.  RNORNS8ZON  OP  CONPUTNR  SCORN 
AGAINST  PRIOR  AVERAGE  SCORN 

Ind. variable  coefficient  std.  error  t-value  sig. level 
C0MP8C0R  29.258444  1.308931  22.3537  0.0000 


R-8Q.  (ADJ.)  -  0.9380 
HAN  «  17.488222 


8N  -  22.407388 
DurbVat  ■  1 . 518 


TABLE  8d.  ANALYSIS  OF  VARIANCE  FOR  THE  FULL  REGRESSION 


Source 

Sun  of  Squares 

DF  Mean  square 

F-Ratio 

P- value 

Model 

250868. 

1  250668. 

499.667 

.0000 

Srror 

16569. 

33  502.09 

Total 

267457. 

34 

R-squared 

•  0.93805 

R-squared 

(Adj.  for  d.f. 

)  -  0.93805 

Stnd.  error  of  est.  ■  22.4074 
Durbin-Vatson  statistic  *  1.51624 


Hypothesis  3,  that  nuaber  of  questions  presented  to  the 
student  in  the  coaputer  exaa  will  be  the  saae  as  the  nuaber 
on  the  written,  was  rejected.  The  written  exaa  consisted  of 
75  questions.  The  nuaber  of  questions  asked  by  the  coaputer 
ranged  froa  9  to  60.  The  average  nuaber  of  questions  asked 
was  29.  A  difference  in  aeana  test  was  perforaed  coaparing 
the  nuaber  of  questions  asked  by  the  coaputer  exaa  with  the 
nuaber  of  questions  in  the  written  exaa  (Table  9).  Results 
of  the  analysis  show  that  H3  is  rejected. 

The  fourth  hypothesis  to  be  tested  was  that  the  tlae 
taken  to  coaplete  the  coaputer  exaa  would  be  the  saae  as  the 
tlae  taken  to  coaplete  the  written  exaa.  The  tiaes  for  both 
groups  on  each  exaa  are  shown  in  Table  10a.  Because  the 
questions  in  the  coaputer  exaa  were  the  saae  as  those  on  the 
written  exaa.  Section  1  students  who  took  the  written  exaa 
first,  had  seen  all  the  questions  the  coaputer  could 
theoretically  ask.  On  the  other  hand  Section  2  students, 
who  had  taken  the  coaputer  exaa  first,  had  only  seen  a 


TABLB  9.  DIFFERENCE  XM  MEANS  BETWEEN  NUMBER 
Or  COMPUTES  ASKED  QUESTIONS 
AMD  WRITTEN  TEST 


MUMQUE8T 


Staple  Nuaber  of  Obs.  34 
Statistics:  Average  29.0SS8 

Variance  101.148 
Std .  Deviation  13.4591 
Median  26 


Confidence  Interval  for  Mean:  95  Percent 

Saaple  1  24.3616  33.756  33  D.P. 

Confidence  Interval  for  variance:  0  Percent 

Saaple  1 


Hypothesis  Test  for  HO: 

Mean  *  75  Coaputed  t  statistic  •  -19.9033 

vs  Alt:  ME  Sig.  Level  ■  0 

at  Alpha  *0.05  so  reject  HO. 


portion  of  the  questions  of  the  written  exaa.  To  avoid  any 
learnlng/aeaory  effects  that  could  have  occurred  between  the 
two  exaas,  the  written  exaa  tiae  of  Section  1  was  coapared 
with  the  coaputer  exaa  tiae  of  Section  2.  A  difference  in 
aeans  test  was  run  on  these  tiaes  and  the  results  are  shown 
in  Table  10b.  Based  on  the  results  of  this  test  H4  was 
rejected. 

A  suaaary  of  the  results  of  this  experiaent  are 
presented  in  Table  11. 
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TABLE  10a.  AVERAGE  EXAM  TIHBS  (minutes) 


Computer  Exam  Time 

Written  Exam  Time 

Section  1 

30 

112 

Section  2 

53 

83 

Combined 

39 

101 

TABLE  10b.  DIFFERENCE  IN  MEANS  BETWEEN  COMPUTER 
AND  WRITTEN  EXAM  TIME8 


CTIMS 

WTIME 

Pooled 

Sample 

Number  of  Obs. 

13 

21 

34 

Statistics: 

Averaqe 

52.7692 

112.095 

89.4118 

Variance 

236.192 

179.89 

201.004 

Std.  Deviation 

15.3685 

13.4123 

14.1776 

Median 

53 

120 

103.5 

Difference  between  Means  >  -59.326 

Conf.  interval  For  Dlff.  In  Means:  95  Percent 

(Equal  Vars.)  Sap 11  -  Sapl2  -69.5199  -49.1322  32  D.F. 
(Unequal  Vars.)  Sap 11  -  Sapl2  >70.0266  -48.6254  22.9  D.F. 

Ratio  of  Variances  ■  1.312983 

Conf.  Interval  for  Ratio  of  Variances:  0  Percent 

Sample  1  v  Sample  2 

Hypothesis  Test  for  HO: 

Dlff  -  0  Computed  t  statistic  -  -11.8573 

vs  Alt:  ME  81q.  Level  •  3.004261-13 

at  Alpha  *0.05  so  reject  HO. 


TABLE  11.  SUMMARY  OP  RESULTS 


HI:  The  scores  achieved  by  a  student  vill  be  the  saae 
Irrespective  of  whether  the  student  takes  the  written 
or  computer  tests. 

MOT  REJECTED 


H2:  The  evaluation  quality  both  of  the  coaputer  and  of 
the  written  exam  will  be  the  saae. 


i 


MOT  REJECTED 

H3:  The  nuaber  of  questions  presented  to  the  student  in 
the  coaputer  exaa  will  be  the  saae  as  the  nuaber  of 
questions  in  the  written  exaa. 


I 


REJECTED 

H4:  The  tiae  taken  to  coaplete  the  coaputer  exaa  will  be 
the  saae  as  than  the  tiae  taken  to  coaplete  the 
written  exaa. 


I 


REJECTED 


k 


VI.  CONCLUSIONS 


This  thesis  presented  the  Interrogative  Diagnostic  Model 
and  an  experiment  designed  to  test  its  validity.  The  model 
provides  an  adaptive  mechanism  to  tailor  testing  sessions  to 
individual  students.  The  results  of  the  experiment  indicate 
that  the  I DM  is  a  reliable  model  in  evaluating  student 
performance.  The  results  also  show  that  the  I DM  effectively 
evaluates  students  in  less  time  and  vith  less  questions  than 
a  written  test. 

The  major  benefits  in  using  such  a  testing  system  of  are 
more  efficient  use  of  students'  and  instructors'  time  and 
less  frustration  in  test  taking  for  both  strong  and  weak 
students.  With  this  type  of  testing  strong  students  are  not 
bored  by  having  to  answer  questions  which  are  beneath  their 
level  of  learning  and  hand  weak  students  are  not  frustrated 
by  having  to  answer  questions  beyond  their  capabilities. 

Further  validation  testing  will  be  needed  before 
widespread  use  is  made  of  this  model.  Possible  future  model 
testing  scenarios  would  be  testing  the  model  in  a  classroom 
situation  in  which  both  computer  tests  and  written  tests  are 
actively  used  to  grade  students.  The  effects  that  computer 
interface  has  on  student  performance  is  also  area  of  related 
research. 
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One  possible  area  in  which  examinations  based  on  the  I DM 
might  prove  beneficial  is  placement  testing  in  industry  and 
the  military.  The  IDM  might  also  prove  valuable  when 
integrated  with  computer-aided  adaptive  instruction. 
However  used,  the  model  can  ultimately  prove  to  be  an 
effective  substitute  for  conventional  diagnostic  methods 
currently  in  use. 


APPENDIX  A 

BAYESIAN  APPROACH  TO  THE  INTERROGATIVE  DIAGNOSTIC  MODEL 

In  applying  Bayesian  analysis  to  the  steps  of  the 
Interrogative  Diagnostic  Model  presented  in  Section  3,  two 
assumptions  are  necessary  [Sivasankaran  1986.  The  initial 
hypotheses  of  step  2  are  derived  by  considering  these  tvo 
assumptions.  The  first  is  that  nothing  is  known  about  the 
subject  and  there  is  a  50-50  chance  that  the  subject  is 
average.  This  posslblity  reflects  complete  uncertianty  and 
becomes  the  uninformed  prior  probablity  pu  =  0.50.  The 
second  assumption  is  that  the  student  has  been  evaluated, 
either  by  a  human  instructor  or  historical  trends,  and  that 
a  better  estimate  on  the  subject  is  available.  This  better 
estimate  Is  termed  pe.  The  degree  of  conviction  in  the 
uninformed  versus  the  estimated  assumptions  is  a:b  vhere  a 
and  b  are  weights.  Expressed  in  probability  notation  this 
becomes 


P(BU)  -  a/(a+b) 

P(Be)  «  b/(a+b) 

where  Bu  denotes  the  strength  of  belief  in  the  uninformed 
opinion  and  Be  the  strength  in  the  belief  in  the  estimated 
opinion.  These  strengths  form  the  Initial  hypotheses  of 
step  2. 


Let  M  be  the  nuaber  of  questions  asked  in  a  given  topic 
area.  The  nuaber  of  questions  answered  correctly  is  C  and 
each  event  is  denoted  B.  The  aatheaatical  foraulation  is 


shown  below. 


P(B/BU)  -  Q  puC  (1  -  pu)  M_C 
P<E/Be>  -  Q  PxC  (1  -  Px>  M”C 


Bayes  theora  P(B/A)  *  P(B)  P(A/B)/[P(B)  P]  aay  now  be 
applied  as  follows: 

P(BU/E)  - 

(a/a+b)  puc  (l-pu)M“C 


< a/a+b)  Q  puC  (l-pu)M"C  +  (b/a+b)  Q  pxC  <l-px)M~C 


and, 

P(BX/E)  «  1  -  P(BU/E) 

where  P(BU/E)  and  P(BX/E)  represent  the  updated  hypotheses 
that  the  student  has  correctly  answered  C  out  of  M 
questions.  Updating  the  hypotheses  is  done  by  steps  3  and  4 
of  the  aodel.  Step  three  asks  the  student  a  question  and 
step  4  calculates  the  updated  hypotheses  based  on  the 
student's  response. 
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Given  the  updated  hypotheises  there  are  two  possible 
courses  of  action.  The  first  is  to  coapare  the  updated 
value  of  P(BX/B)  with  upper  and  lover  cutoff  criteria.  The 
upper  cutoff  criteria,  alpha,  represents  the  minimum 
strength  of  belief  in  the  estimated  opinion  that  is  required 
to  conclude  that  the  students  knowledge  is  good.  If  P(BX/E) 
exceeds  alpha,  the  student  is  judged  knowledgeable  enough  in 
the  topic  area  being  evaluated  that  no  further  questions  are 
needed.  Conversely,  the  lover  cutoff  criteria,  beta, 
represents  the  minimum  strength  of  belief  in  the  estimated 
opinion  required  to  discern  weak  students.  If  P(BX/B)  does 
not  exceed  beta,  the  student  is  judged  unknovledgeable  in 
the  topic  area  being  evaluated  and  no  further  questions  in 
that  topic  area  are  needed. 

The  second  course  of  action  is  necessary  when  all  the 
questions  in  the  critical  level  have  been  asked  and  the 
updated  hypothesis  P(BX/B)  lies  between  alpha  and  beta.  In 
this  case  the  student  is  considered  to  have  an  average 
knowledge  of  the  subject  area  and  the  model  then  proceeds  to 
the  lover  level  of  supplementary  questions  in  the  same  topic 
area  or  if  all  the  supplemental  questions  have  been  asked, 
the  model  proceeds  to  a  new  topic  area. 

I DM  step  5  compares  P(BX/E),  the  revised  estimated 
opinion,  vith  the  cutoff  values  and  if  neither  of  the  cutoff 
criteria  have  been  been  reached  the  model  returns  to  step  4. 
If  one  of  the  criteria  has  been  reached  or  there  are  no 
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questions  remaining  to  be  asked  in  the  in  the  cognitive 
level,  the  model  proceeds  to  step  6.  Step  6  accepts  or 
rejects  the  updated  hypothesis  based  on  the  cutoff  criteria. 
If  student  has  exceeded  the  upper  cutoff  or  has  not  exceeded 
the  lover  cutoff,  the  hypothesis  that  the  student  is  average 
is  rejected.  This  means  that  the  student  has  been  evaluated 
as  either  surerior  or  poor  and,  in  either  case,  the  model 
proceeds  to  a  nev  topic  area  by  returning  to  step  one.  If 
the  hypothesis  is  accepted  step  7  will  ask  further 
supplementary  questions  to  further  define  the  students 
knowledge  level  or,  if  the  supplenentary  questions  have  been 
asked,  will  proceed  to  a  nev  topic  area. 


APPENDIX  B 
PROGRAM  LISTING 


This  appendix  contains  a  listing  to  the  dBASB  III  Plus 
program  files  which  implementated  the  I DM.  The  appendix 
begins  with  a  list  of  the  memory  variables  and  datafield 
names  used  in  the  program  files.  The  programs  IQ1P1.PRG  and 
IQ1P2.PRG  are  an  example  of  the  programming  method  used  to 
implement  the  two  screen  critical  questions  mentioned  in 
Chapter  IV.  The  program  files  using  this  method  are 
referred  to  as  QUBSTION  in  the  program  structure  chart  of 
Figure  3. 


MEMORY  VARIABLES 
variable  Representing 

aa  degree  of  conviction  in  uninformed  probability 

aalfa  threshold  for  beliving  subject  is  above  average 


bb 

bbeta 

f 

1 

k 

kf 

moreq 

nf 


degree  of  conviction  in  expert  probability 
threshold  for  beliving  subject  is  below  average 
value  of  combinations  returned  from  factor. prg 
counter  in  factor. prg 

temporary  storage  variable  for  (numasked  -  right) 

temporary  storage  variable  for  kl 

flag  indicating  if  further  supplemental  questions 
are  needed 

temporary  storage  variable  for  numasked t 
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nuaasked 

nuacrit 

nuuup 

PBUB 

PBXB 

PBBU 

PBBX 

rf 

right 

SBU 

SBB 

Stop 

teap 

wrong 

u 

x 


nuaber  of  questions  asked  thus  far  of  a  certian 
topic  and  type 

nuaber  of  critical  questions  of  a  given  type 

nuaber  of  suppleaentary  questions  of  a  given  type 

Bayesian  probability  of  the  uninforaed  opinion 
being  correct  given  the  event 

Bayesian  probability  of  the  estiaated  opinion 
being  correct  given  the  event 

probability  of  the  event  given  the  uninforaed 
opinion 

probability  of  the  event  given  the  estiaated 
opinion 

teaporary  storage  variable  for  right  I 

nuaber  of  questions  answered  froa  a  given  type 
and  topic  area 

strength  of  belief  in  uninforaed  opinion 

strength  of  belief  in  estiaated  opinion 

flag  Indicating  if  further  critical  questions 
are  needed 

teaporary  storage  variable  used  in  f actor. prg 

nuaber  of  questions  answered  incorrectly  froa  a 
given  type  and  topic  area 

uninforaed  prior  probability  that  a  subject 
is  not  average 

expert's  prior  probability  that  a  subject  is 
above  average 


DAT AFIELD  NAMES 


Pie Id  Names  Remarks 

A  degree  of  conviction  in  uninformed  probability 

ALFA  threshold  for  beliving  subject  is  above  average 

ANSWER  the  answer  to  a  particular  question 

AREA  the  topic  area  of  questions  being  asked 

ASTART  the  time  the  subject  was  presented  a  question 

AFINI8H  the  time  the  subject  answered  the  question 

B  the  degree  of  conviction  in  expert  probability 

BETA  threshold  for  beliving  subject  is  below  average 

CUTOFF  minimum  percentage  of  supplemental  questions  the 
program  will  present  to  the  subject 

NAME  subjects  last  name 

LEVEL  difficulty  level  of  a  particular  question 

PR06RAM1  the  first  page  program  of  a  critical  question 

PROGRAM 2  the  second  page  program  of  a  critical  question 

PU  uninformed  prior  probability  that  a  subject 

is  not  average 

PX  expert's  prior  probability  that  a  subject  is 

above  average 

QNUM  question  identifier 

QUE8TI0N  memo  entry  containing  the  question  and 
multiple  choice  answers 

SRESPONSE  the  subjects  answer  to  the  question 

SSN  last  four  digits  of  a  subject's  SSN  for 

identification  purposes 

SUP  name  of  the  DBF  file  containing  the  supplemental 

questions 

TYPE  cognative  domain  level  of  question  being  asked 


O',  tiA-SfcfriJZ 


*  Program. . 

*  Author . . . 

*  Date . : 

*  Notice. . . : 

* 

*  Notea . . . . : 


MAIN.PRG 
PAT  ODONNELL 
SPRING  1988 

Copyright  (c)  1988,  PAT-SIV,  All  Rights 

Reserved 
THESIS 


SET  TALK  0F7 
8BT  BELL  OFP 
SET  E8CAPB  ON 
SET  CONFIRM  ON 
SET  COLOR  TO  W+/B 

*  This  is  the  main  program  module  for  a  computer  aided 

*  testing  program.  This  module  calls  initilaizes  the 

*  criterion  values,  and  then  calls  tvo  other  modules 

*  which  do  the  work  of  the  program. 


aa  ■  0 
bb  *  0 
aalfa  =  0 
bbeta  *  0 
areanum  =  1 
X  -  0 
U  »  0 

ccutoff  *  0 

USE  criterion 

STORE  ALFA  TO  aalfa 

8TORE  BETA  TO  bbeta 

8TORE  A  TO  aa 

STORE  B  TO  bb 

STORE  PX  TO  x 

STORE  PU  TO  u 

STORE  CUTOFF  TO  ccutoff 

USE 


SELECT  1 
USE  blkboard 

SELECT  2 
USE  dbarea 


CLEAR 

TEXT 
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WELCOME  TO  THE 


COMPUTER  AIDED  TESTING  SECTION 
07  THE  18-4183  PINAL  EXAMINATION 


ENDTBXT 
§  22/10 

WAIT 


CLEAR 
DO  INFO 


STORE  "  "  TO  aarea 
SELECT  2 
GO  TOP 

DO  WHILE  areanua  <=  3 
STORE  area  TO  aarea 
DO  ASK 
SELECT  2 

areanua  *  areanua  +  1 
SKIP 
BNDDO 

CLEAR 

TEXT 

ENDTBXT 

EOF  MAIN . PRG 


*  Prograa. . 

*  Author . . . : 

*  Data . 

*  Notice. . . : 

* 

*  Motes . . . . : 


INPO.PRO 
PAT  ODONNELL 
SPRING  1988 

Copyright  (c)  1988,  PAT-SZV,  All 

Reserved 
THESIS 


Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 


*  This  program  module  introduces  the  program  to  the 

*  student, gives  instructions  on  using  the  prograa  and 

*  gets  the  student's  name  and  last  four  digits  of  his 

*  SSN  for  scoring. 

CLEAR 

TEXT 


Good  morning  and  welcome  to  an  experiment  in 
computer  aided  testing.  The  purpose  of  this  experiment 
is  to  try  and  determine  whether  a  computer,  using  a 
small  bank  of  questions  and  Bayesian  analysis,  can 
determine  an  Individual's  knowledge  level  in  a  given 
subject  area  as  effectively  as  a  traditional  written 
exam.  The  prograa  presented  here  is  not  a  mere  test 
with  a  fixed  number  of  questions  to  be  answered.  This 
prograa  adjusts  both  the  number  and  type  of  questions 
to  the  individual.  Should  this  experiment  prove 
successful,  testing  programs  of  this  type  could  have 
broad  application  in  educational,  industrial  and 
military  environments. 

BNDTEXT 

#22,10 

WAIT 

CLEAR 

TEXT 


INSTRUCTIONS:  You  will  be  shown  a  series  of  questions 
and  the  prograa  will  prompt  you  for  the  answer.  Just 
type  in  your  answer  followed  by  a  carriage  return.  Some 
questions  are  presented  in  two  screens.  You  may  page 
back  and  forth  between  screens  by  following  the  system 
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prompts.  If  you  have  any  questions  there  vill  be 
someone  in  the  lab  to  answer  them. 

IMPORTANT:  Be  aware  that  you  must  answer  each  question 
in  order,  in  other  words  you  cannot  skip  a  question  and 
return  to  it  later. 

Professor  Sivasankaran  will  use  the  hiqher  score 
between  this  computer  aided  test  and  your  written  exam, 
so  this  does  "count." 

Good  luck! 

BNDTEXT 
8  22,10 
WAIT 

CLEAR 

SELECT  1 

MNAME  =  '  • 

MSSN  =  0000 

8  10,10  SAY  (Enter  your  lastname  and  last  four  digits 

of  your  1 

8  11,10  SAY  (social  security  number  in  the  appropriate 
blocks.  ] 

8  14,10  SAY  "  LASTNAME:  " 

8  14,33  GET  MNAME  PICTURE  " 11! Ill  I  III !!!!!!!!!! " 

READ 

8  16,10  SAY  "  SSN:  " 

8  16,29  GET  MSSN  PICTURE  "fill" 

READ 

APPEND  BLANK 

REPLACE  NAME  WITH  MNAME,  SSN  WITH  MSSN 
RETURN 


*  EOF  INFO.PRG 


*  Program..:  ASK.PRG 

*  Author...:  PAT  ODOMMBLL 

*  Date . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-Siv,  All  Rights 

*  Reserved 

*  Notes....:  THBSXS 

SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 


*  This  program  asks  the  student  a  series  of  critical 
and  complementary  questions  from  the  topic  database 
file.  The  program  forwards  the  results  of  the 
questioning  to  calculation  modules  for  bayesian 
analysis. 

nuasup  *  00 
numcrit  »  5 
nuaasked  *  00 
stop  «  'NO' 


right 

■ 

00 

moreq 

S 

•NO' 

STORE 

N 

"  TO 

supp 

STORE 

II 

C"  TO 

type 

8TORB 

1 

'  TO 

start 

STORE 

« 

*  TO 

finish 

SELECT  3 
USE  taarea 
GO  TOP 

STORE  sup  to  supp 

DO  WHILE  .NOT.  BO F< )  .AND.  stop  -  'NO' 

STORE  programl  TO  pagel 
STORE  prograa2  TO  page 2 
STORE  "  "  TO  response 

goback  ■  "Y" 

DO  WHILE  goback  <>  "N" 

DO  DOIT 
ENDDO 

8  24,05  SAY  "Your  answer  to  question  " 

8  24,29  SAY  page2 

8  24,32  SAY  "  is:  (one  of  a,  b,  c  ...]" 
8  24,  59  get  response  picture  'I' 

READ 

8  24,05  SAY  " 


You  chose  " 


II 


0  24,25  SAY  response 

6  24,26  SAY  "  Press  ENTER  to  confirm 
9  24,25  GET  response  PICTURE  'J* 

READ 

STORE  TIHEO  TO  finish 

IP  response  ■  answer 
right  *  right  +  1 
ENDIP 

SELECT  1 

APPEND  BLANK 

REPLACE  qnua  WITH  page 2 

REPLACE  sresponse  WITH  response 

REPLACE  astart  WITH  start 

REPLACE  afinish  WITH  finish 

SELECT  3 

nuaasked  *  nuaasked  +  1 

DO  correct 

SKIP 

BNDDO 


IF  aoreq  -  'YES' 

SELECT  4 
use  asupp 
GO  TOP 

DO  WHILE  .NOT.  EOF( ) 

IP  type  -’S' 

nuasup  *  nuasup  +  1 
ENDIP 
SKIP 
ENDDO 

Stop  ■  'NO' 
nuaasked  *  0 
right  ■  0 

STORE  '  '  TO  suppnua 
GO  TOP 

DO  WHILE  .NOT.  EOFO  .AND.  Stop  *  'NO* 

IP  type  *  'S' 

STORE  STR(RBCNO( ) ,  2,  0)  TO  suppnua 
CLEAR 
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STORE  TIHEO  TO  start 


e 

3, 10 

? 

•> 

n 

"z  QUESTION 

* 

STORE  ' 

'  TO  response 

8 

22,20 

SAY 

"Your  answer  to  question 

8 

22,44 

SAY 

supp 

8 

22,45 

SAY 

"8Q  " 

8 

22,41 

SAY 

suppnua 

8 

22,45 

SAY 

"  is  ■ 

8 

22/53 

GET 

response 

PICTURE  "l" 

READ 

8 

22/10 

SAY 

M 

You  chose  M 

8 

22/30 

SAY 

response 

8 

22/31 

SAY 

"  Press 

ENTER  to  confira 

8 

22/30 

GET 

response 

PICTURE  ' l • 

READ 

STORE  TIMEO  TO  £inish 

IF  response  *  answer 
right  =  right  +  1 
ENDIF 

SELECT  1 
APPEND  BLANK 

REPLACE  qnun  WITH  suppnua 
REPLACE  sresponse  WITH  response 
REPLACE  astart  WITH  start 
REPLACE  af inish  WITH  finish 
SELECT  4 

nuaasked  =  nuaasked  +  1 
DO  correct 
SELECT  4 

ENDIF 

SKIP 

ENDDO 

ENDIF 

RETURN 

*  eof  ask.prg 
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*  Program..:  IQ1P1.PRG 

*  Author...:  PAT  ODOMMELL 

*  Date . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-SIV,  All  Rights 

*  Reserved 

*  Notes. . . . :  THESIS 


CLEAR 

TEXT 


Consider  a  STUDENT  file  with  the  following  fields  used 
in  a  file  processing  system: 


Name 

character 

15 

Rank 

character 

6 

Address 

character 

20 

City 

character 

10 

state 

character 

2 

Zip 

numeric 

5 

Salary 

numeric 

5 

Age 

numeric 

2 

One  of  the  records  in  the  STUDENT  file  contains  the 
following  values: 

Smith  LCDR  202FranklinSt  Monterey  CA  93940  42000  33 

ENDTEXT 
%  24,10 
WAIT 
RETURN 

•EOF  IQ1P1.PRG 


48 


*  Program. . :  IQ1P2.PRG 

*  Author . . . :  PAT  ODONNELL 

*  Date . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-SIV,  All  Rights 

*  Reserved 

*  Notes _ :  THESIS 

CLEAR 

TEXT 


If  the  Postal  authorities  decided  to  increase  the  zip 
code  length  to  six  digits  and  the  application  program  vas 
so  modified,  John's  salary  (based  on  the  existing 
data  files)  would: 

a.  not  change  at  all 

b.  would  increase  by  $3 

c.  would  increase  by  $100,003 

d.  would  increase  by  $30,000 

e.  would  decrease  by  $21997 

f.  would  decrease  by  $40000 

g.  None  of  the  above  is  true 


BNDTEXT 

RETURN 

•EOF  IQ2P2.PRG 
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*  Program..:  DOIT.PRG 

•  Author...:  PAT  ODONNBLL 

•  Date . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-SIV,  All  Rights 

*  Reserved 

•  Notes....:  THESIS 

SIT  TALK  OPP 
8ST  BSLL  OPP 
SBT  CONPIRM  ON 

*  This  is  the  program  module  which  presents  the 

*  critical  questions  to  the  student. 


STORE  "  "  TO  goback 

STORE  TIMEO  TO  start 
DO  apagel 
DO  &page2 

%  24,5  SAY  "Do  you  vant  to  go  back  to  the  previous  page? 
(Y  or  N)" 

8  24,59  GET  goback  picture  "!N 

READ 

RETURN 

*eof  DOIT.PRG 


50 


*  Program. . :  CORRECT. PRO 

*  Author...:  PAT  ODOMMELL 

*  Date . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-SIV,  All  Rights 

*  Reserved 

*  Notes....:  THESIS 

SET  TALK  OPP 
SET  BELL  OPP 
SET  CONPIRM  ON 

*  This  program  module  calculates  the  Bayesian 

*  probabilities  of  the  student's  correct  answers. 

CLEAR 
f  -  0 
eval  ■  0 

do  PACTOR  with  numasked,  right,  £ 

SBU  ■  aa/(aa  ♦  bb) 

8BX  -  bb/(aa  +  bb) 

PEBU  »  f  *  (u  **  right)  *  ((1  -  u)  **  (numasked  -  right)) 

PEBX  *  f  *  (x  **  right)  *  ((1  -  x)  **  (numasked  -  right)) 

PBUE  -  (SBU  *  PEBU)  /  ((SBU  *  PEBU)  +  (SBX  *  PEBX)) 

PBXE  »  1  -  PBUE 

*814,10  say  "PBXE  -  " 

•814,18  say  PBXE 
*820,10 
•WAIT 

CLEAR 

DO  CASE 

CASE  type  -  'C'  .AND.  PBXE  >■  aalfa 

stop  *  'YES' 
eval  ■  1 
DO  tally 

CASE  type  *  'C'  .AND.  PBXE  <«  bbeta 

stop  »  'YES' 
eval  *  5 
DO  tally 

CASE  type  *  'C'  .AND.  numasked  ■  numcrit 
moreq  *  'YES' 

CASE  type  ■  'S' 

IP  (numasked/numsup)  >«  ccutoff  .AND.  PBXE  >*  aalfa 


Stop  -  ' YES ' 
eval  «  2 
DO  tally 

ENDXF 

IF  ( nuaasked/nuasup )  >*  ccutoff  .AND.  PBXB  <«  bbeta 
stop  -  ’YES' 
eval  ■  3 
DO  tally 

ENDIF 

CASE  type  ■  ’S’  .AND.  nuaasked  *  nuasup 
eval  ■  4 
DO  tally 

BNDCASB 

RETURN 

*  eo£  correct.prg 
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*  Program..:  TALLY. PRO 

*  Author...:  PAT  ODONHBLL 

*  Data . :  8PRIMG  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-SIV,  All  Rights 

*  Reserved 

*  Notes _ :  THIS  IS 

SET  TALK  OPP 
SET  BELL  OPP 
SET  CONPIRM  ON 

*  This  is  the  program  module  vhich  appends  the  area 

*  evaluations  to  the  blkboard  file. 


SELECT  1 
APPEND  BLANK 
REPLACE  name  WITH  aarea 
REPLACE  ssn  WITH  aval 
SELECT  3 

RETURN 


*  EOP  TALLY . PRG 


*  Program. . ;  FACTOR . PRO 

*  Author...:  PAT  O'DONNELL 

•  Data . :  SPRING  1988 

*  Notice...:  Copyright  (c)  1988,  PAT-81 V,  All  rights 

•  Reserved 

•  Notes....:  THESIS 

PARAMETERS  nuaasked,  right,  £ 

SET  TALK  OFF 
SET  BELL  OFF 

teap« 1 
i  -  1 

do  vhile  i  <■  numasked 
temp  ■  temp  *  i 
i  «  i  +  1 
enddo 
nf  *  temp 

temp  *  1 
i  -  1 

do  vhile  i  <■  right 
temp  -  temp  •  i 
i  »  i  +  1 
enddo 
rf  ■  temp 

temp  «  1 
i  -  1 

k  =  numasked  -  right 

do  vhile  i  <■  k 
temp  »  temp  *  i 
i  -  i  ♦  1 
enddo 
kf  «  temp 

£  «  n£/(k£  *  r£) 


•  EOF  FACTOR. PRO 


APPSNDIX  C 
QUSSTIOM  LISTING 


This  appendix  lists  the  75  questions  contained  in  the 
written  and  computer  final  exams.  The  questions  are  divided 
into  the  three  topic  areas:  introduction,  normalization  and 
management.  Sach  topic  area  begins  with  five  critical 
questions  followed  by  twenty  supplementary  questions.  The 
difficulty  level  of  the  question  is  indicated  by  a  number 
from  one  to  three.  This  difficulty  level  was  determined  by 
the  expert  opinion  of  the  instructor  based  on  his  experience 
of  the  questions  in  previous  quarters.  The  meaning  of  those 
numbers  is  as  follows: 

1  less  difficult 

2  average  dlffculty 

3  very  difficult 

The  answer  to  each  question  is  indicated  by  >  or  A. 


omaiiflMfi 


Introduction  Critical  Question  1  Level  3 

Consider  a  STUDENT  file  with  the  following  fields  used 
in  a  file  processing  systea: 


Mane 

character 

15 

Rank 

character 

6 

Address 

character 

20 

City 

character 

10 

State 

character 

2 

Zip 

nuaerlc 

5 

Salary 

numeric 

5 

Age 

nuaerlc 

2 

One  of  the  records  in  the  STUDENT  file  contains  the 
following  values: 

Snith  LCDR  202FranklinSt  Monterey  CA  93940  42000  33 


If  the  Postal  authorities  decided  to  increase  the  zip 
code  length  to  six  digits  and  the  application  progran  was 
so  Modified,  John's  salary  (based  on  the  existing 
data  files)  would: 

a.  not  change  at  all 

b.  would  increase  by  $3 

c.  would  Increase  by  0100,003 

d.  would  increase  by  $30,000 

>  e.  would  decrease  by  $21997 

£.  would  decrease  by  $40000 

g.  None  of  the  above  is  true 


Introduction  Critical  Question  2 


Level  2 


Which  one  of  the  following  portrays  accurately  a  subset 
of  the  functional  components  of  a  DBMS? 


1. 


I  I  Logical 

I  Definition  I  I/O  request 

I  Tools  l< - 

I  Subsystem  I 
I _ I 


— >—  I 


I 


Logical 
I  Processing  I  I/O  request 

I  interface  I < - > 

I  Subsystem  I 


I. 


.1 


II. 


1 

Data 

1 

1 

Design 

l<- 

1 

Subsystem 

1 

1 

1 

1 

1. 

.1 

Logical  1 

1 

DBMS  1 

Physical 

I/O  1  — 

----> | 

|< 

— >  I/O 

requests  I 

1 

engine  1 

request 

1 

Data 

1 

1 

1 

1 

1 

administration 

1 

1 

1 

1 

Subsystem 

l<* 

III. 


I  I  Logical 

I  Data  I  I/O  Reguest 

I  Dictionary  l< - 

I  I 

I _ I 

a.  I  and  II  only 

b.  II  and  III  only 
>  c.  I  and  III  only 

d.  Ill  only 

e.  all  the  above 


1 

Physical 

1 

DBMS 

I/O  request 

>1 

engine 

1 

< - >,  | 

1  Op  1 
1 System 1 

1 

< - >,  | 

1 

DB 

I/O 

Commands 

57 


Introduction  Critical  Question  3 


Level  3 


The  Prise  Area  of  a  storage  that  uses  index  sequential 
Method  contains  the  following  records  (in  unblocked  fornat): 


Track  1 

RXC  5 

RXC  10 

RBC  15 

RBC  20 

Track  2 

RBC  25 

RXC  30 

RBC  35 

RBC  40 

Track  3 

RXC  45 

RXC  50 

RBC  55 

RXC  60 

If 

RBC 

22  and  RBC  23 

i  were  to 

be 

added 

i,  the  new 

track 

index 

would  : 

Look 

like 

a. 

20 

20 

null 

b. 

1 

20 

22 

address 

of 

22 

2 

40 

40 

null 

2 

40 

40 

null 

3 

60 

60 

null 

3 

60 

60 

null 

>  c. 

1 

20 

20 

null 

d. 

1 

20 

22 

address 

of 

22 

2 

30 

40 

address 

of 

35 

2 

40 

35 

address 

of 

35 

3 

60 

60 

null 

3 

60 

60 

null 

e. 

1 

20 

20 

address 

of 

22 

2 

40 

40 

address 

of 

33 

3 

60 

60 

null 

Introduction  Critical  Question  4 


Level  1 


Consider  the  following  object  diagraa: 


COMPUTER 

STUDENT 

CLASS 

Computer!  1 

1  Student! 

1 

1  Cnase 

1 

Osver  1 

1  Sphone 

1 

1  Tothrs 

1 

Acqdate  j 

1  CLASS (MV) 

1 

I  STUDENT (MV) 

1 

Status  I 

1  APPOINTMENT (MV) 

1 

1 

J 

Station!  1 

1 

SOFTWARE ( MV ) 1 

APPOINTMENT 

1 

SOFTS AR* 

I  Apptdate 

1  Snaae  1 

1  Apptstrt 

1  Copy!  1 

1  CLASS 

1  APPOINTMENT (MV) | 

1  STUDENT 

1  COMPUTBR(MV)  I 

1  COMPUTER 

1  1 

I  SOFTWARE (MV) 

I 


The  Bachman  diagram  for  the  above  objects  is  given  by: 


I. 


->  COMPUTER  << 


■»  SOFTWARE  << - 1 

I 


II. 


I . 

I 

I - >STUDEMT« - 

- >  COMPUTER  <<■ 


-»  APPOINT  «« 
•»  MENT  <-■ 


I 

-»CLASS< - 1 


»  SOFTWARE  << - 1 

I 


III 


I - 

I 

I - >  STUDENT  <<- 

- >  COMPUTER  <<■ 


— >  APPOINT  «■ 
■>>  MENT  «• 


I 

>>  CLASS  < - 1 


»  SOFTWARE  « - 1 

I 


I - 

I 

I - >  STUDENT  «- 


->  APPOINT  «■ 
■>>  MENT  <— 


■>>  CLASS  < - I 


a.  only  I  is  true 

b.  only  II  is  true 
>  c.  only  III  is  true 


d.  only  II  and  III  are  true 

e.  all  are  true 

f.  none  are  true 


Introduction  Critical  Question  5 

Consider  the  following  table: 

Scheduling  Inventory 

Students  X 

Class  Rosters  X  X 

Computers 

Faculty 


Level  1 


Usage  Analysis 

X 

X 

X 


Which  of  the  following  is  true? 

I.  Column  I  indicates  the  different  applications 
in  a  database  system  and  the  remaining  columns 
Indicate  files  used  by  a  DBMS. 


II 


”XN  indicates  which  file  is  used  by  which  DB 
application. 


III.  Bach  column  in  the  table  is  called  a  "schema". 

IV.  Column  1  indicates  the  different  files  and  the 
remaining  columns  indicte  applications. 

V.  The  table  is  called  a  "schema". 

a.  Ir  II,  IV  and  V  only 
>  b.  II,  IV  and  V  only 

c.  II  and  IV  only 

d.  II,  III  and  IV  only 

e.  All  the  above  are  true 


Introduction  Supplementary  Question  1  Level  2 

The  disadvantages  of  file  processing  systems  are: 

I .  data  is  separated  and  isolated 

II.  data  is  often  duplicated 

III.  application  programs  are  dependent  on  file 
formats 

IV.  processing  is  slower  than  manual  systems 

a.  all  the  above 
>  b.  I,  II  and  III  only 

c.  II/  III  and  IV  only 

d.  I  and  III  only 


Introduction  Supplementary  Question  2  Level  3 

The  functions  of  a  DBMS  engine  include: 

I.  build  conceptual  schema  structure 

II.  manage  disc  and  memory  space 

III.  provide  object  and  domain  definitions 

IV.  map  logical  I/O  reguests  into  physical  activity 

a.  I  and  III 
>  b.  II  and  IV 

c.  I,  II  and  IV 

d.  all  of  the  above 

e.  none  of  the  above 


Introduction  Supplementary  Question  3  Level  3 

Whether  a  property  is  single-  or  multiple-valued  has 
nothing  to  do  with  whether  it  is  an  object  or 
nonobject  property. 

T  or  F 

A 

Introduction  Supplementary  Question  4  Level  2 

Database  data  can  be  updated  in  different  ways  by 
user: 

I.  entering  data  via  a  load  utility 

II.  entering  data  with  a  form 

III.  entering  data  in  a  tabular  format 

IV.  entering  data  through  database  views 


a.  all  of  the  above 

b.  II,  III  and  IV 
>  c.  I,  II  and  III 

d.  I,  II  and  IV 


Introduction  Supplementary  Question  5  Level  2 

Some  of  the  functions  of  a  DBMS  are: 

I.  store,  retrieve  and  update  data 

II.  provide  object  and  Bachman  diagrams 

III.  provide  coordination  and  control  facilities  for 
concurrent  processing 

IV.  provide  facilities  for  backup  and  recovery 

a.  all  the  above 

b.  II  and  IV 

c.  I,  II  and  IV 
>  d.  I,  III  and  IV 


Introduction  Supplementary  Question  6  Level  3 

Integrity  rules  are  very  easy  to  enforce  using  a  DBMS 

T  or  F 

A 

Introduction  Supplementary  Question  7  Level  3 

The  domain  of  an  object  property  is  a  set  of  object 
instances . 

T  or  F 


Introduction  Supplementary  Question  8  Level  3 

The  fact  that  a  property  has  multiple  values  does  not 
imply  that  it  must  be  an  object  property. 

T  or  F 


Introduction  Supplementary  Question  9  Level  2 

An  OBJECT  is  a  named  collection  of  determinants  that 
sufficiently  describes  an  entity  in  the  users  vork 
environment. 

T  or  F  < 
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Introduction  Supplementary  Question  10 


Level  2 


A  set  of  attributes  with  the  property  that  no  two  tuples 
In  the  relation  can  ever  have  the  same  values  of  these 
attributes  is  called  the  relation  key. 

T  or  P 

a 


Introduction  Supplementary  Question  11  Level  2 

The  Portion  of  an  object  that  is  visible  to  a  particular 
application  is  called  a  VIEW. 

T  or  P 

A 


Introduction  Supplementary  Question  12  Level  2 

A  complex  relationship  between  two  relations  can  be 
represented  by  including  the  key  of  one  relation  as  an 
attribute  in  the  other. 

T  or  P 

a 


Introduction  Supplementary  Question  13  Level  1 

The  components  of  a  database  application  are: 

I .  hardware 

I I .  programs 

III.  data 

IV.  procedures 

V.  personnell 

>  a.  all  the  above 

b.  I,  II,  III  and  V 

c.  I,  II  and  III 

d.  Ill,  IV  and  V 


Introduction  Supplementary  Question  14  Level  1 

Some  types  of  database  data  are: 

I.  application  data 

II.  database  metadata 

III.  overhead  data 

IV.  transaction  log  data 
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a.  all  o£  the  above 
>  b.  I,  II  and  III 

c.  I  and  III  only 

d.  II  and  IV  only 

Introduction  8uppleaentary  Question  15  Level  1 

The  different  phases  of  database  systea  developaent  are: 

I.  definition 

II.  requlreaent 

III.  evaluation 

IV.  design 

V.  iapleaent 


a.  I ,  II,  III  and  IV 

b.  I,  III,  IV  and  V 

c.  IV  and  V 

>  d.  all  of  the  above 


Introduction  Suppleaentary  Question  16  Level  1 

Different  attributes  aay  take  values  froa  the  saae 
doaain. 

T  or  P 

a 


Introduction  Suppleaentary  Question  17  Level  1 

Bach  record  in  a  relation  is  unique. 

T  or  P 

A 


Introduction  Suppleaentary  Question  18  Level  1 

k  relation  is  a  flat  file. 

T  or  P 

A 


Introduction  Suppleaentary  Question  19  Level  1 

To  be  able  to  join  two  relations  there  aust  always  be 
one  coaaon  attribute. 

T  or  P 
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Introduction  Supplementary  Question  20 


Level  1 


The  SKLBCT  operation  defines  a  subset  of  tuples  of  a 
given  relation  (i.e.  horizontal  slice)  based  on  a  Boolean 
condition. 

T  or  P 


QU18TIQMS 


(formalization  Critical  Question  1  Level  3 

Consider  the  relation  vith  the  following  structure: 

8TUDBMTJU8&OB  (SID,  Compute r_Mumber ,  Class,  Sname ,  TotalJHours) 


You  are  also  given: 


81D  - >>  Cosputer_Nusber 

SID  - >>  Class 

8ID  - >  Snase 

8  ID  - >  Total_Hours 


Snase  - >  8ID 


Which  of  the  following  is  false? 

I.  SID  and  Snase  are  determinants  but  SID  alone  Is  the  key 

II.  There  is  at  least  one  transitive  dependency  leading  to 
anomolles 

III.  There  is  exactly  one  MVD 

IV.  The  relation  can  be  upgraded  to  the  next  higher  normal 
form  by  decomposing  it  into  smaller  relations 


>  a.  I,  II  and  III 

b.  II,  III  and  IV 

c.  I,  II  and  IV 

d.  all  the  above 

e.  none  of  the  above 


Normalization  Critical  Question  2 


Level  3 


Consider  the  following  three  relations  with  the  given 
functional  dependencies.  Identify  the  normal  form  in  which 
each  relation  is  in. 


R1 

<A, 

c. 

B, 

D) 

k  - > 

B 

R2 

(A, 

B/ 

B) 

kC - > 

D 

R3 

(A, 

c. 

D, 

P) 

k  - > 

B  - > 

D  - > 

B 

CD 

P 

a.  R1  -  2ndNP 

>  b.  Rl  -  IstNP 

R2  -  3rdNP 

R2  -  BCNP 

R3  -  2ndNP 

R3  -  2ndNP 

c.  Rl  -  3rdNP 

d.  Rl  -  IstNP 

R2  -  BCNP 

R2  -  3rdNP 

R3  -  2ndNP 

R3  -  2ndNP 

e.  none  of  the  above 


Normalization  Critical  Question  3  Level  3 

Consider  the  relation  8_J_T  with  three  objects  respectively 
Student,  Subject  and  Teacher.  The  following  constraints  apply: 

—  for  each  subject,  each  student  of  that  subject  is 
taught  by  only  one  teacher 

—  each  teacher  teaches  only  one  subject 

—  each  subject  is  taught  by  several  teachers 
Which  of  the  following  is  true? 

I.  S_J_T  is  in  BCNP 

II.  S_J_T  has  MVDs 

III.  S_J_T  has  one  possible  composite  key 

a.  I  only  d.  all  of  the  above 

b.  II  only  >  e.  none  of  the  above 

c.  Ill  only 
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Normalization  Critical  Question  4 

When  these  two  relations  are  joined. 


Level  1 


i 

PART 

SUPPLIER 

't 

'A 

p» 

COLOR 

CITY 

8#  Pft 

1 

pi 

red 

LA 

81  PI 

1 

P2 

blue 

NY 

SI  P2 

1 

P3 

green 

SF 

82  PI 

1 

P4 

blue 

NY 

Which  one  o£  the  following  is  true? 

a.  there  will  be  exactly  12  tuples  in  the  resultant  join 

>  b.  there  will  be  exactly  3  tuples  in  the  resultant  join 

c.  there  will  be  exactly  4  tuples  in  the  resultant  join 

d.  there  will  be  exactly  7  tuples  in  the  resultant  join 

e.  none  of  the  above 


Noraalization  Critical  Question  5 
Oiven  the  relations  below: 
CABLE  CUSTOMER 
I  I 

Figure  1  I  Account*  I — O - 0 — 

I _ I 

SHIPMENT 

I  I  I 


Level  1 


CONVERTOR 


1 

1  1  1 

1  serial# 

I...I  Account# 

1 

1  1 

PARCEL 

1  1 

Parcel# 

1 . . . 1  Container 

1  1 

PART 


BIN  LOCATION 


Figure  3 


I  III  III 

I  Part#  I...I  Bin#  l—O - 1  — I  Bin*  I...  I 

I _ I _ I _ I  I _ I _ I 


Figure  4 


PLAYER 

I  I  I 

I  Naae  I  ...  I  —  I 
I _ I _ 1/ 


POSITION 

i  r 

I  — I  Position  I. 
\l _ l_ 
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Which  of  the  following  is  true? 

I.  Pig  1  represents  an  optional -optional  relationship 

II.  Pig  2  represents  a  mandatory-optional  relationship 

III.  Pig  3  represents  an  optional-mandatory  relationship 

IV.  Pig  4  represents  a  mandatory-mandatory  relationship 


>  a.  I  and  IV  only 

b.  II  and  III  only 

c.  I,  II  and  III  only 

d.  I,  II  and  IV  only 

e.  all  the  above  are  true 


Normalization  Supplemental  Question  1  Level  3 

A  loss  projection  occurs  when 

>  a.  false  data  is  generated  by  a  projection 

b.  a  projection  loses  attribute  values 

c.  a  projection  loses  tuples 

d.  a  projection  cannot  be  made  on  two  specific 
relations 


Normalization  Supplemental  Question  2  Level  3 

Which  one  of  the  following  is  true  in  one  to  one 
relationships? 

I.  Attributes  that  have  a  one  to  one  relationship  must 
occur  together  in  at  least  one  relation,  (call  the 
relation  R  and  the  attributes  A  and  B) 

II.  Both  A  and  B  must  be  part  of  the  key  of  R. 

III.  An  attribute  can  be  added  to  R  if  it  is  functionally 
determined  by  A  or  B. 

IV.  A  and  B  must  occur  together  in  R,  but  should  not  occur 
together  in  other  relations. 

a.  I ,  II  and  III 
>  b.  I,  III  and  IV 

c.  II  and  III 

d.  all  of  the  above 


Normalization  Supplemental  Question  3  Level  3 

When  evaluating  database  design 

a.  machine  efficiency  has  highest  priority 

b.  elimination  of  modification  anomalies  has  the 
highest  priority 

c.  relation  independence  has  the  highest  priority 
>  d.  priorities  will  be  different  depending  upon 

requirements 

Normalization  Supplemental  Question  4  Level  2 

Primary  keys  must  be  unique  within  a  relation  but 
candidate  keys  do  not  have  to  be  unique  within  a  relation. 


normalization  Supplemental  Question  5  Level  3 

In  the  following  functional  dependancy  A  ->  B  ->  c, 

a.  the  dependancy  is  a  multivalued  dependancy 
>  b.  C  is  a  transitive  dependancy  with  A  as  its 
determinant 

c.  C  is  the  determinant 

d.  k  and  C  are  not  dependent 


normalization  Supplemental  Question  6  Level  2 

If  a  relation  is  in  domain  key  normal  form,  it 

>  a.  could  have  no  modification  anomalies 

b.  may  still  have  transitive  dependencies 

c.  could  still  have  join  dependencies 

d.  could  still  have  multivalued  dependencies 


normalization  Supplemental  Question  7  Level  3 

A  relation  in  first  normal  form  can  generally  be  placed 
in  second  normal  form  by 


>  a. 

projections 

b. 

joins 

c. 

selections 

d. 

products 

normalization  Supplemental  Question  8  Level  3 

An  interrelation  constraint  exists  when 

>  a.  two  or  more  relations  share  the  sane  attribute 

b.  two  attributes  in  the  relation  have  the  same 
determinant 

c.  two  determinants  in  a  relation  have  the  same 

subject  attribute 

d.  two  or  more  attributes  of  a  candidate  key  have 
different  determinants 


normalization  Supplementary  Question  9  Level  3 

A  modification  anomaly  occurs  when  change  to  one  entity 
causes  a  change  to  a  second  entity. 

T  or  P  < 
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Normalization  Supplemental  Question  10  Level  3 

Multivalued  dependencies  occur  vhen 

a.  several  attributes  are  dependent  upon  the 
same  determinant 

b.  a  single  attribute  has  several  determinants 

c.  there  are  several  transitive  dependencies 

>  d.  several  Independent  attributes  are  dependent 
on  one  other  attribute 


Normalization  Supplemental  Question  11  Level  3 

Which  of  the  following  is  not  a  relational  design 
evaluation  criterion: 

a.  elimination  of  modification  anoaolles 

b.  relation  independence 
>  c.  ease  of  use 

d.  reduction  of  key  attributes 


Normalization  Supplemental  Question  12  Level  3 

Attributes  that  have  a  many  to  many  relationship  can 
exist  together  in  a  single  relation  (although  this  would 
create  redundancies).  Assume  two  such  attributes  B  and  F, 
residing  together  in  a  relation  T. 

Which  of  the  following  is  false? 

I .  The  key  of  T  must  be  B 

II.  The  key  of  T  must  be  F 

III.  Both  B  and  F  can  be  candidate  keys 

IV.  An  attribute  can  be  added  to  T  if  it  is  determined 
by  B  or  F 

a.  I  only 

b.  II  only 

c.  Ill  and  IV  only 
>  d.  all  the  above 

e.  none  of  the  above 


Normalization  Supplemental  Question  13  Level  1 

A  relation  is  in  domain/key  normal  form  when 

a.  the  domain  of  the  key  is  the  function  of  the  key 

b.  the  key  is  the  function  of  the  domain 

c.  it  has  no  multivalued  dependencies 

>  d.  every  constraint  is  a  logical  consequence  of 
the  definition  of  keys  and  domains 


Normalization  Supplemental  Question  14  Level  1 

If  a  relation  is  in  third  normal  form,  it  must  be  in 
second  normal  fora. 

T  or  F 


Normalization  Supplemental  Question  15  Level  1 

A  relation  is  in  Boyce-Codd  normal  fora  if 

a.  it  has  only  one  key 

b.  it  has  no  transitive  dependencies 

>  c.  every  determinant  is  a  candidate  key 

d.  there  are  no  deletion  anomalies 


Normalization  Supplemental  Question  16  Level  1 

If  a  tuple  cannot  be  inserted  into  a  relation  until  an 
additional  fact  about  another  entity  is  known,  it  is 
referred  to  as: 

a.  a  deletion  anomaly 
>  b.  an  insertion  anoaoly 

c.  a  functional  dependency 

d.  an  insertion  redundancy 


Normalization  Supplemental  Question  17  Level  1 

If  a  relation  meets  the  criterion  that  all  nonkey 
attributes  are  dependent  upon  all  of  the  key,  it  must  be  in: 

>  a.  second  normal  form 

b.  fourth  normal  form 

c.  domain/key  nornal  form 

d.  Boyce-Codd  normal  fora 
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Normalization  Supplemental  Question  18  Level  1 

In  the  following  functional  dependency  A  ->  B  ->  C,  the 
determinant (s) : 

a.  is  A  only 

b.  is  C  only 

>  c.  are  A  and  B 
d.  are  B  and  c 


Normalization  Supplemental  Question  19  Level  1 

Multivalued  dependencies  always  exist  in  pairs. 

T  or  P 


Normalization  Supplemental  Question  20  Level  1 

If  an  attribute  is  a  key  it  must  be  a  determinant. 

T  or  P 


MANAGEMENT  QUESTIONS 


Management  Critical  Question  1 


Level  3 


You  are  given 
database  crashed. 

Relative 

Record 

Number 


this  transaction  log  at  the  time  the 


1 

OT1 

0 

2 

11 

42 

START 

2 

OT1 

1 

4 

11 

43 

MODIFY 

CU8T100 

(Old 

value ) 

(new  value) 

3 

OT2 

0 

8 

11 

46 

START 

4 

OT1 

2 

5 

11 

47 

MODIFY 

SP  AA 

(Old 

value) 

(nev  value) 

5 

OT1 

4 

7 

11 

47 

INSERT 

ORDER  11 

(value) 

6 

CT1 

0 

9 

11 

48 

START 

7 

OT1 

5 

0 

11 

49 

COMMIT 

8 

OT2 

2 

0 

11 

50 

COMMIT 

9 

CT1 

6 

10 

11 

51 

MODIFY 

SP  AA 

(old 

value) 

(nev  value) 

10 

CT1 

9 

0 

11 

51 

COMMIT 

Which 

of  the 

transactions  vlll  be 

reprocessed 

after  the 

last  save? 


I .  0T1 

II.  0T2 

III.  CT1 


a.  I  only 

b.  II  only 

c.  Ill  only 

d.  all  of  the  above 
>  e.  none  of  the  above 
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Management  Critical  Question  2 


Level  2 


There  are  three  users  in  a  multiuser  DB. 
perform  the  following  sequence  of  events. 


Each  vants  to 


USER  A 


USER  B 


USER  C 


Read  rec  100 
Change  rec  100 
Write  rec  100 


Read  rec  200 
Change  rec  200 
Write  rec  200 


Read  rec  200 
Change  rec  200 
Write  rec  200 


Which  of  the  following  sequences  violates  serializability? 


I.  II.  III. 

Read  100  for  A  Read  200  for  B  Read  200  for  C 

Read  200  for  B  Read  100  for  A  Change  200  for  C 

Change  100  for  A  Change  100  for  A  Read  100  for  A 

Write  100  for  A  Write  100  for  A  Write  200  for  C 

Read  200  for  C  Change  200  for  B  Change  100  for  A 

Change  200  for  B  Read  200  for  C  Read  200  for  B 

Write  200  for  B  Change  200  for  C  Change  200  for  B 

Change  200  for  C  Write  200  for  C  Write  100  for  A 

Write  200  for  C  Write  200  for  B  Write  200  for  B 


Select  your  ansver: 

>  a.  I  and  II  only 

b.  II  and  III  only 

c.  I  and  III  only 

d.  Ill  only 


Management  Critical  Question  3 


Level  3 


Pive  records  are  stored  physically  on  a  disc  track  as 
shown  below: 

RRN  4 

I _ I 


/ 

\ 

RRN  5  / 

\  RRN  3 

/ 

\ 

__  / 

\  _ 

1  (a  very 

1 

1  crude 

1 

disc  track) 

1  RRN  2 

1 

1 

\ 

l_ 

/ 

\ 

/ 

\ 

/  RRN  1 

\ 

/ 

The  actual  contents  of 

each  record 

is  given  in  the 

following 

table : 

RPN 

SUPNAME 

PARTI D 

PRICE 

1 

Topnotch 

P003 

9.95 

2 

Ugo 

P014 

0.99 

3 

Bestparts 

P020 

29.95 

4 

Acne 

P008 

99.99 

5 

Gotcha 

P002 

11.00 

Assume 

you  access  the 

data  in  the 

following  ways: 

I .  Access 

the  entire  file  in  the  order  specified 

by  a 

linked 

list  created 

according 

to  ascending 

order 

(alphabetic)  by  Supnaae. 

I I .  Access 

the  entire  file  in  the  order  specified 

by  a 

linked 

list  created 

according 

to  ascending 

order 

(numeric)  by  Part ID. 

Assuming 

the  disc  rotates  in  a  clockwise  direction 

9 

which  of  the  following  is 

true? 

>  a.  There  will  be  no  difference  in  the  number  of  disc 
rotations  if  the  entire  file  were  to  be  accessed  by 
either  I  or  II. 

b.  Method  I  will  be  faster  by  one  rotation  than  aethod  II. 

c.  Method  II  will  be  faster  by  one  rotation  than  aethod  I. 

d.  Method  I  will  be  faster  by  two  rotations  than  method  II. 

e.  Method  II  will  be  faster  by  two  rotations  than  aethod  I. 


Management  Critical  Question  4 


Level  2 


The  following  table  is  an  example  of 

PATIENT  Record 

Subjects  Who  Know  Subjects  Who  Know 

Password  SICK  Password  PAIN 

Y  Y 

Y  N 

Y  N 

Y  N 

N  N 


Read 
Insert 
Actions:  Modify 

Delete 
Grant 


a.  Fernandez,  Summers,  Wood  model  of  database  security 

b.  subject-orientated  security 
>  c.  entity-orientated  security 

d.  all  of  the  above 


Management  Critical  Question  5 


Level  1 


You  are  given  the  following  table: 


Relative 

Record 

Student- 

Class- 

Number 

number 

number 

Semester 

1 

200 

70 

88S 

2 

100 

30 

89P 

3 

300 

20 

897 

4 

200 

30 

888 

5 

300 

70 

88S 

6 

100 

20 

888 

Deteraine  vhich 

one  is  the 

correct  inverted  list  (or  index) 

I. 

II. 

III. 

Relative 

Relative  i 

Relative 

Student- 

Record 

1  Class- 

Record  t 

Record 

number 

Number 

1  number 

Number  1 

Semester 

Number 

100 

2 

1  20 

3  1 

88S 

1 

100 

6 

1  20 

6  1 

88S 

4 

200 

1 

1  30 

2  1 

88S 

5 

200 

4 

1  30 

4  1 

888 

6 

300 

3 

1  70 

1  1 

897 

2 

300 

5 

1  70 

5  1 

897 

3 

a.  I  only 

b.  II  only 

c.  Ill  only 

d.  I  and  II  only 
>  e.  all  the  above 


Management  Suppleaental  Question  1  Level  1 

A  binary  relationship  involves  at  least  tvo  record 
types. 


T  or  P 

a 


Management  Supplemental  Question  2  Level  3 

When  the  relationship  is  a  l:n  and  is  mandatory  in  both 
directions,  there  is  a  strong  likelyhood  that  the  records 
are  describing  different  aspects  of  the  same  object. 

T  or  P 

A 


Management  Supplemental  Question  3  Level  3 

An  object  relationship  is  one-to-one  if: 

I .  object  k  contains  object  B  as  a  single-valued 
object  property 

II.  object  B  contains  Object  A  as  a  single¬ 
valued  object  property 

III.  object  B  does  not  contain  object  A 


a.  only  condition  I  is  necessary 
>  b.  conditions  I  and  II  alone  are  sufficient 

c.  conditions  II  or  III  alone  are  sufficient 

d.  conditions  I,  II  and  III  are  all  necessary 


Management  Suppleaental  Question  4  Level  3 

If  A  is  related  to  B  and  B  is  not  related  to  A,  it  is 
referred  to  as  0:1  relationship. 

T  or  F 

A 

Management  Supplemental  Question  5  Level  3 

The  aulti -valued  attributes  in  a  composite  object  never 
represent  another  object. 

T  or  P 

A 


Manageaent  Suppleaental  Question  6 


Level  3 


The  keys  of  tvo  relations  having  a  compound-object 
relationship  between  them  do  not  have  a  common  attribute. 


T  or  P 


Management  Supplemental  Question  7  Level  1 

An  atomic  transaction  is 

a.  any  set  of  activities  that  operate  on  the  same 
data- items 

>  b.  a  set  of  activities  that  form  one  logical  unit  of 

vork 

c.  any  set  of  activities  that  can  be  performed  vith 
one  instruction  to  the  DBMS 

d.  any  set  of  activities  that  can  be  performed 
concurrently 

Management  Supplemental  Question  8  Level  2 

To  roll  forward, 

>  a.  restore  the  database  from  a  save  and  apply  after 

Images 

b.  use  the  current  copy  of  the  database  and  apply 
the  after  images 

c.  restore  the  database  and  apply  before  images 

d.  use  the  current  copy  of  the  database  and  apply 
before  images 


Management  Supplemental  Question  9  Level  3 

A  check  point  record  is  written 


a.  after  all  updates  have  been  made  to  the  database 


b.  after 

all 

requests 

have 

been 

written 

to 

the 

transation 

log 

>  c.  after 

all 

requests 

have 

been 

written 

to 

the 

database  and  the  transaction  log 
d.  before  updates  have  been  written  to  the  database 


Management  Supplemental  Question  10 


Level  1 


In  the  Fernandez,  Summers,  and  Vood  Model  of  database 
security,  a  subject  can  be 

>  a.  the  program  that  will  be  processing  the  database 

b.  the  element  of  the  database  that  will  be  protected 

c.  the  processing  permissions  given  to  a  user 

d.  the  user's  password 


Management  Supplemental  Question  11  Level  2 

Constraints  on  user's  activity  can  be  enforced  by 

a.  subject -oriented  security 

b.  object  oriented  security 

c.  password  matrices 
>  d.  user  exits 


Management  Supplemental  Question  12  Level  1 

Functions  of  a  DBA  include: 

I .  Management  of  data  activity 

II.  Management  of  database  structure 

III.  Database  Personnel  and  Placement 

IV.  Management  of  the  DBMS 

a.  I,  II,  IV  only 

b.  II  and  IV  only 

>  c.  All  of  the  above 

d.  IV  only 


Management  Supplemental  Question  13  Level  1 

Concurrent  processing  implies  that  the  DBMS  processes 
the  queries  of  various  users  simultaneously. 

T  or  F 

* 


Management  Supplemental  Question  14  Level  2 

Ser iallzabillty  ensures  that  results  of  two  concurrent 
transactions  are  the  same  as  they  would  have  been  had  they 
been  processed  one  at  a  time. 

>  T  or  P 
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Management  Supplemental  Question  15 


Level  1 


Database  changes  that  have  been  made  by  a  transaction  in  j 

process  are  said  to  be  uncommitted. 

T  or  P 


Management  Supplemental  Question  16  Level  1 

Concurrent  processing  does  not  lead  to  lost  updates  if 
transactions  are  only  reading  data. 

T  or  P 

A 

Management  Supplemental  Question  17  Level  1 

Deadlock  occurs  when  each  o£  two  transactions  is  waiting 
for  a  resource  the  other  has  locked. 

T  or  P 


Management  Supplemental  Question  18  Level  1 

Checkpoints  are  snapshots  of  updated  parts  of  a 
database . 

T  or  P 

A 


Management  Supplemental  Question  19  Level  1 

The  Performance  Monitor  maintains  the  configuration 
control  of  the  DBMS. 

T  or  F 

A 


Management  Supplemental  Question  20  Level  1 

Interleaving  means  that  while  two  or  more  transactions 
are  processing  concurrently  the  computer  system  executes 
some  instructions  from  one,  then  executes  some  from  the 
other,  switching  back  and  forth  between  them. 

T  or  P 

A 


I 

j 


APPENDIX  D 

COMPUTER-AIDED  TB8T  QUESTIONNAIRE 


LAST  NAME _  LAST  POUR  DIGITS  OP  SSN 


1.  What  are  your  general  feelings  toward  this  coaputer 
aided  testing  procedure? 

very  soaevhat  soaevhat  very 

negative  negative  indifferent  positive  positive 


2.  How  do  you  feel  that  not  being  able  to  explain  your 
answers  affects  the  accuracy  of  the  test? 

very  soaevhat  soaevhat  very 

negatively  negatively  indifferently  positively  positively 


3.  How  do  you  feel  about  not  having  to  answer  essay 
questions? 

very  soaevhat  soaevhat  very 

negatively  negatively  indifferently  positively  positively 


4.  Did  you  feel  aore  pressure  vith  the  coaputer  test  than 
you  have  usually  felt  during  other  written  exaas? 

Yes  or  No 


5.  Rate  your  coafort  level  in  using  coaputers. 


very 

uncoafortable 


indifferent 


soaevhat 

uncoafortable 


very 

coafortable 

soaevhat 

coafortable 


6.  Do  you  think  a  coaputer  aided  exaa  tests  the  knowledge 
of  the  student  aore  effectively  than  a  written  exaa 

Yes  or  No 
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7.  Do  you  think  that  aaking  such  prograas  available  foe 
practice  would  help  students  learn  subject  aatter 
better? 

Yes  or  Mo 


8.  Please  note  any  coaaents,  likes/dislikes  or 
recoaaendations  you  have  about  this  test  or  the 
concept  of  coaputer  aided  testing  in  general? 


APPENDIX  B 
EXPERIMENT  DATA 


STUDENT 

SECTION 

BXAM1 

BXAM2 

PINAL 

1 

1 

40 

43 

56 

2 

1 

49 

47 

76 

3 

1 

48 

43 

59 

4 

1 

46 

46 

46 

5 

1 

50 

42 

51 

6 

1 

46 

41 

59 

7 

1 

49 

45 

64 

8 

1 

39 

41 

64 

9 

1 

42 

48 

74 

10 

1 

47 

47 

60 

11 

1 

45 

45 

74 

12 

1 

44 

37 

45 

13 

1 

48 

44 

70 

14 

1 

48 

47 

67 

15 

1 

45 

46 

58 

16 

1 

47 

45 

74 

17 

1 

49 

43 

68 

18 

1 

46 

40 

52 

19 

1 

43 

43 

38 

20 

1 

33 

45 

55 

21 

1 

34 

38 

45 

22 

2 

47 

48 

72 

23 

2 

43 

42 

57 

24 

2 

48 

49 

65 

25 

2 

44 

46 

75 

26 

2 

46 

43 

60 

27 

2 

39 

45 

55 

28 

2 

44 

44 

61 

29 

2 

43 

46 

51 

30 

2 

48 

48 

72 

31 

2 

49 

46 

75 

32 

2 

43 

41 

68 

33 

2 

38 

43 

51 

34 

2 

39 

43 

48 

COMPSCOR 

2.34 

4 

3.34 
2 
2 

3 

3.34 

2.67 

4 

2.34 
3 
2 

3 

3.34 
2 

3.67 

4 
2 
1 

1.67 

1.67 

3.34 

2.34 

4.34 

3.34 
3 

3.34 

3.67 

2.34 

3.34 

3.34 

3.34 

2.34 

1.34 
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STUDENT 

QUSST1 

QUEST2 

QUEST 3 

QUBST4 

QUEST 5 

QUBST6 

GENLATT 

XPLNANS 

NOBSSYQ 

PCVDPRES 

COMFLVL 

PCVDEFCT 

1 

0 

-2 

0 

1 

2 

1 

2 

0 

0 

2 

1 

4 

0 

3 

-2 

-2 

-4 

0 

4 

0 

4 

2 

4 

4 

1 

4 

0 

5 

-2 

-4 

-4 

0 

4 

0 

6 

2 

0 

-2 

1 

2 

0 

7 

2 

-2 

0 

0 

2 

0 

8 

-2 

0 

-2 

0 

-2 

0 

9 

0 

-2 

2 

1 

4 

0 

10 

0 

0 

2 

0 

2 

11 

4 

2 

2 

1 

0 

0 

12 

2 

-4 

4 

1 

2 

1 

13 

2 

-2 

-2 

0 

4 

0 

14 

2 

0 

4 

1 

4 

0 

15 

2 

2 

2 

1 

2 

0 

16 

4 

-2 

-2 

4 

1 

17 

4 

2 

4 

1 

4 

1 

18 

2 

0 

4 

1 

0 

0 

19 

-4 

0 

0 

-2 

0 

0 

20 

0 

-2 

0 

1 

2 

0 

21 

2 

2 

4 

1 

-4 

0 

22 

4 

-2 

4 

1 

4 

1 

23 

24 

0 

-2 

2 

1 

4 

0 

25 

-4 

-2 

4 

0 

4 

0 

26 

-4 

-4 

0 

0 

4 

1 

27 

0 

-2 

4 

0 

4 

0 

28 

-2 

2 

2 

0 

-2 

0 

29 

-2 

-2 

0 

0 

-2 

0 

30 

2 

-2 

4 

0 

4 

0 

31 

-2 

-2 

0 

0 

-2 

0 

32 

-2 

0 

0 

0 

4 

1 

33 

34 

2 

2 

4 

0 

4 

0 
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STUDENT 

QUEST7 

CTIME 

(MIN) 

VTIME 

(MIN) 

NUMQUBST 

1 

1 

46 

120 

29 

2 

1 

28 

120 

45 

3 

1 

12 

120 

17 

4 

1 

27 

120 

18 

5 

0 

18 

75 

19 

6 

1 

32 

107 

25 

7 

1 

35 

119 

45 

8 

0 

19 

103 

21 

9 

1 

15 

120 

33 

10 

1 

40 

120 

26 

11 

1 

55 

120 

60 

12 

1 

25 

114 

41 

13 

1 

26 

117 

35 

14 

1 

29 

104 

23 

15 

1 

22 

120 

40 

16 

0 

41 

120 

46 

17 

1 

25 

90 

38 

18 

1 

19 

85 

20 

19 

0 

17 

120 

9 

20 

1 

45 

120 

27 

21 

1 

56 

120 

26 

22 

1 

63 

100 

17 

23 

26 

80 

9 

24 

1 

53 

70 

25 

25 

0 

48 

85 

52 

26 

0 

70 

113 

60 

27 

1 

44 

95 

20 

28 

1 

61 

94 

29 

29 

1 

77 

53 

27 

30 

1 

56 

78 

24 

31 

0 

67 

112 

36 

32 

1 

50 

70 

17 

33 

44 

60 

9 

34 

1 

27 

70 

20 
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